Qwen3.8-Flash-Next (qwen.ai)

Yeah 122B is the sweet spot for me as well. Even deepseek flash overthinks on stuff way too much. I think they fully rely on large reasoning turns to achieve better quality. The result of course means we wait a long time to get results even with high throughput as a lot of tokens are wasted.

You made me irrationally laugh reading this

This is glitch art for text, I love it

On one hand I love your joke, on the other, this is HN not reddit and I usually downvote such responses, not sure what is the HN etiquette for such humor?

I reached point three and was nodding all along. I guess I am the NPC

That's a caveat, and a real one.-

  ⎿  You've hit your session limit · resets 2:50am (123°24′W Etc/GMT+8)
  /upgrade to increase your usage limit.

> Three honest possibilities, and I’ll be straight with you about each

This. I don't know if the "honest answer" phrasing is part of the system prompt or alignment, but when people say "honestly" all the time I start wondering how honest they're being.

I killed my own ssh session twice with pkill -f, because the pattern matched the command line containing it.

> You’re absolutely right to be hopeful. Three honest possibilities, and I’ll be straight with you about each:

> [UGC styled humorously as LLMisms]

All joking aside, having interacted with Claude intensely for the last 8 months and about 30 hours/week in the last 3, I’ve started to notice how (for want of a better word) “readable” (“digestible” ? “comprehensible” ? “Predictable” is the wrong direction.) information chunked into LLM-shaped pieces are for me.

I can digest LLM-shaped pieces of data very easily probably because I’ve been spending too much time with Claude, sure.

But the other side of this is that the entire human species (using LLMs) is similarly being trained to digest interrelated pieces of information/data in these specific shapes, akin to how philosophical assertions can be formulated as a syllogism and, thus, become more readily understood because of familiar epistemological cadence and shape.

Many people reject such copy/prose/data because they detect AI-generated-so-not-worth-human-attention, but I do wonder if this is preparing many millions of loosely (and tightly) associated humans and their organizations to quickly exchange and digest information.

This is not to say current LLMisms are the end, only that such detectable patterns in information delivery will make comprehension and communication more efficient (as well as more limited precisely because of such structure).

/philosophical musings about the epistemological implications of LLM-shaped conversation tics

This might be the most angry I've ever been at a HN comment that I upvoted

Yeah I understand, it's my assumption that the actually/wait/but have a point. It doesn't reduce the fact that it increases the time for tasks substantially.

> For example, even if you make thinking tokens literally just

Generally speaking yes, but actually no (just randomness is suboptimal, adding steps just to add steps is suboptimal). There is a mechanism working there (in having a CoT) that is not quite clear.

The task is to optimize the efficiency of CoT. Understanding that it is not a plain "chain of thought" is the start of the problem, the solution is not there yet.

If we had the solution, there would exist no overthinking - CoT would be optimal (lean and essential plus best results).

Especially on practical tasks. One shot prompts work better at Q6_K_XL for me. It loads a file, then analyses then second guesses itself then again then again then it tries to come up with a solution then second guess rinse and repeat. 122b is the perfect balance but it lacks quality for harder to solve stuff. I've ran DS Flash 0731 at Q4KXL, 3.8 Q6KXL, GLM 5.2 Q4KXL and they all over-reason. At least that's how it looks like to me when comparing with frontier models, even weaker ones.

Yeah, I ran into an overthinking loop with it a couple days ago on a task that shouldn't have been that hard. (It's kind of interesting to watch the internal conversation happening with it). Overall I'm impressed with it, but setting the /effort to medium is what you usually want (it defaults to xhigh). I do wonder if I had made it write out a plan if I would have avoided that though.

I use medium generally, that's about a minute at 20t/s and off for general chat (few seconds for a response). What kind of setup are you running it on?

This is needs ~80GB of fast memory at 4 bits per weight. Faster memory is better, but probably even something like 3090 + 64GB RAM should work (not fast, but maybe even 20-30 t/s? llama.cpp support pending).

If/when we can get larger context this will mostly be mitigated by these smaller models being able to search the internet.

Self-learning/improving would be even better but that's still a long way to go.

For world knowledge, you'd want it to find and reference the source material to be sure. At that point, it doesn't matter if the knowledge is embedded.

In a lot of spaces, this is actually preferable.

Ex - nodejs natively supports a huge set of typescript with built-in type stripping these days. But ask most hosted models to build a typescript project and they default to a heavy compile step, or a tool like tsx, ts-node, etc.

Models with lots of "world knowledge" have a good chunk of that knowledge go stale, and there's no real way to refresh it without training a new model.

Another classic example of this back in the day was to ask who the president of the US was, and watch different models happily give different answers based on the date they were trained.

---

Personally, I'm really interested to see if we're headed towards a spot where the model is entirely distinct from the knowledge store.

We're vaguely there with the ability for models to go search the web, but I think the reliability of that path is going to continue declining (more and more spam content, less and less genuine value).

I kinda want a paradigm where I can pick and engine and a knowledge bank, and combine them as I please.

Ex - if I'm doing gardening, I can pick "gardening for models (version 32)" as my knowledge store.

If I'm doing auto-repair... "cars for dummies (version 3)". etc...

At 125B + 51B I'd expect it to have some degree of world knowledge, clearly in the middle between small models like qwen 27B, and huge trillion parameter models.

128GB, 4-bit quantized.

This is a good counter argument. But you have to note that this is after OpenAI cut Luna costs by 80%. If you compare launch pricing, Qwen probably comes out ahead on a cost-performance basis.

Those prices are just tokens? Since each model uses different amounts of tokens to do the same thing, it's a misleading price that often makes open-weights look more competitive than they are, since most open weights models use dramatically more tokens and time to complete tasks than many frontier models.

In Artifical Analysis's cost per task, Luna(max) costs $0.05 per task, and Qwen 3.8 27B costs $0.25 per task, a 5X increase. We'll see how 3.8-flash-next does.

Sounds like discrete propaganda

I believe you're underestimating the lag inherent in the economy. Even if we grant the idea that the political party controlling the US House/Senate has a significant impact on the economy, and that the current party is BAD and the next one would be GOOD, I would still expect that things will continue getting WORSE for a good 4 to 8 years before they get better again.

And that's even with assuming that we can continue to ignore the long-term problems like social security insolvency, the debt bomb, or climate change forever.

You know the memory cartel isn't even close to being broken, right?

They are already working on it.

https://huggingface.co/unsloth/Qwen3.8-Flash-Next-GGUF https://unsloth.ai/docs/models/qwen3.8-next

> You will need at least 75 GB of RAM or unified memory to run the model. Its smallest 1-bit quantized version is larger than usual because of the model’s architecture so 1-bit isn't really 1-bit at all. However, this also means the quantization is less aggressive, allowing the model to retain more of its original accuracy than more heavily quantized models.

Lots of RAM required even for the 1-bit, which is already downloadable. Interested to see how well this one works compared to Ornith1.5-35B-A3B I've been running (and quite happy about).

Edit: but llama-cpp does not yet support it.

I have this working on a branch of my https://github.com/rdaum/eider (for DGX Spark)

nVME paging the n-gram table (in BF16 for now).

Still working at it. Prefill sucks still but decode is about 12 tok/sec and the model weights fit nicely in the 128GB Spark memory in nvfp4 quant while paging the ngram stuff from disk.

If Simon would pitch for example PCBWay that and I am pretty sure they will sponsor it (assuming their logo stays). They can do laser engraved versions also ;)

In their page they say it will need at least 112GB[0], so including context, that would be a tight fit. I'm also hoping I can make a q4 fit on my 128GB strix halo

[0]: https://unsloth.ai/docs/models/qwen3.8-next#qwen3.8-flash-ne...

That probably includes the 51b ngrams too. It's possible that those could be streamed from NVMe on-demand. The Engram paper that developed this technique streamed from RAM to VRAM at only ~1% performance degradation, but these strix halo boxes and the spark have much slower memory, so it's possible moving down another rung on the memory hierarchy wouldn't affect their performance too much.

This will almost certainly require changes to llama.cpp or vllm to do it right.

ah I don't think that page was up when I checked, it 404ed!

I can run Deepseek flash 0731 version (ds4, esl3) on single DGX spark. getting around ~20 tok/s. It's great. Quantized version of this model would probably run on the DGX spark. I am excited to wait for quantized models that fits in single DGX spark.

180B?

0.03 / 0.075 ? Where can i get that prices? Especially during peak hours DS4flash became much more money hungry than last month.

https://api-docs.deepseek.com/quick_start/pricing

Deepseek 0732 is $0.22/$0.66 off peak