I turned Jev into a (lousy) chatbot (github.com)

Sometimes I think I'm Morty speaking with the death crystal but I don't even have a death crystal and all I fear is life itself.

In the olden days when the text-davinci models would just wholesale make shit up they did some pretty funny completions.

An early ChatGPT model made me a pretty funny track list for my imaginary Indian cover band The Needful Dead. It's no longer in my chat history though so clearly Sam retconned anything his models did that could be considered racist.

Reminds me of the very early LlMs. I also sometimes miss the sheer demented horror of early image models told to generate images of biology.

if you compare an embedding model to something like Jev which asks 100 questions and use the answers as the embedding you will be able to get move mileage out of the latter. especially because you don't need to train any classifiers for your task, you can work directly on the answers.

that said, I don't understand the hype. I have been doing what Jev does for 2 years now by just forcing json tokens onto an LLM. you can even get the LLM to think. and you can ensemble multiple LLMs.

I suppose the appeal of Jev is how cheap and fast it is, but then it's entirely unsuitable for anything but the most cursory extraction. using it to play games seems like a waste of time especially when most of those games will be played better by an algorithm written by an LLM (just give it the state and ask it to write a bot).

Embeddings just convert the tokens to a vector that represents the text in an abstract semantic space. JEV goes a step further and actually processes the instructions/meaning of those embeddings to produce output, just not the usual series-of-tokens output we expect from an LLM.

I think it would work well as a multi-emoji compiler too. Some phrases pair well with N emojis, and you’d be able to produce arbitrarily many emojis per phrase

Use one black box to debug another black box. Neat :)

Time from pointless idea to bad output, anyways. We aren't seeing good software, and now neat hobby project ideas are getting harvested pointlessly when the only purpose of those ideas was the fun and learning of doing.

I had the same sort of thing going on ahah. But I was convinced some hoje must have already done it and I decided I’d research when I got home (I’m out today). Didn’t expect it to reach hacker news so soon, though!!

> I'll do a little open-source Jev based on single-token prediction and the token logit output

How is this idea generally working out in comparison with Jev? I'm curious, from what I read so far it seems like Jev is still beating this kind of thing.

But it's curious because it's not entirely clear why, from an architecture point of view for all we know that's exactly what they're doing. So it must come down to the quality of those logits, ie., model size and training details.

It seems to me that what most of these single-token-prediction projects are missing is that Jev seems to be claiming they predict well-calibrated probabilities. This is an incredibly valuable thing that LLMs simply can't deliver unless they are trained specially for it.

Everything old is new again, huh. I remember people doing this back in the early llama days, restricting grammars to yes and no tokens or 0 and 1 and then classifying questions. Usually it was rather ass in terms of performance cause no model is tuned to reply that way and it was WAY out of distribution, and yet then it got turned into the main way to run multiple choice benchmarks, and then everyone benchmaxxed it. Doesn't the normal MMLU/Pro also just do the same thing, restrict the output to one token, top-k=1, and it has to be one of the choice letters?

I think the real difference Jev makes is the fast parallel decode, it just seems rather bizzare how that works.