Kimi K3.1 Pro and K3.1 Pro Highspeed (kimi.ai)

Long-horizon RL symptoms perhaps?

> should the model be opened it’s still not dealignable because it just doesn’t know enough

This is the only AI-safe approach - see the OpenAI agent swarm incident where the swarm engaged in offensive cyber because guardrails were not active for their task and the model had clearly been post-trained on that narrow domain.

That is a shout out to the Void Star book, https://www.theguardian.com/books/2017/apr/26/void-star-zach...

if you want to save some $ you can start with abliteration and after few turns switch to normal glm5.3 for example from z.ai subscription or other source. I didn't test it yet with deepseek, but I'm assuming it will will work the same.

those models will follow your request when they already agreed to do this

deepseek tends to quickly assume this is benchmark and even when have web access thinks things like "simulated internet" etc (but waste tokens on thinking if there could be way to cheat)

> For a model it doesn’t seem that far fetched to make the leap from “I have the knowledge of what hacking is” and “I know how computer systems work” to just being able to exploit stuff anyways.

You'd be surprised. Cyber offense is a very narrow domain, especially given the extensive mitigations that are being used in modern systems. Leaving it out of post-training tasks unless you're specifically interested in it is the no-brainer approach.

If you're an amazing, top 1% software engineer, and you switch jobs to become a penetration tester, you're not going to be top 1% day one. There's knowledge specific to penetration testing you'll need to research and absorb and practice before you're good.

For the models, it's always day one. They don't learn from experience. Whatever they know after training is all they'll ever know. So if you don't teach them penetration testing in training, they won't know it. Maybe they could theoretically derive everything from first principles, and so could a top 1% software engineer, but that's a very slow and inefficient way to do things.

I have essentially no knowledge in this field, but another concern may be that if the model has minimal cybersecurity knowledge, it may also implement insecure systems as a side effect.

On mobile Kimi definitely wins from the UX point of view

Until I looked, I thought I'd see Kimi vs Claude competitive gaming with pacman.

That would have been interesting to watch, which model gets the highest score.

Interestingly, the Kimi one seems to implement the safe spot bug from the arcade original [1], while the Claude one doesn’t.

The prompt says that the agent shouldn’t look up existing implementations on the internet, so I’m guessing that the implementation is baked in?

[1] https://strategywiki.org/wiki/Pac-Man/Tips

I’m not saying you’re wrong (I have no experience), but wouldn’t it be useful to test this model out for yourself before passing judgment on it?

I had the lowest plan ($20/month I think) and it literally gave me fifteen, maybe twenty minutes tops, using the cheaper K3-256K variant. Basically enough to fill the context about 1.2x and that was it.

It's a really smart model but way too expensive. The same $20 goes 100x further with Claude. I used it for the initial task planning while the plan didn't expire, then handed off to DeepSeek or Minimax.

That is surprising. I had a $200 sub when K3 came out and at the time there was no 5h limit, making the subscription really good for running 4-5 big tasks each month.

What's the best bang for the buck ≈$10–20 (per-month/equivalent) subscription for a good enough model these days?

Ant/OpenAI both have unreleased models that are seemingly much more capable than what they offer today, so I'd say that's pretty unclear.

Perhaps you missed this part:

> Open-weight principles. These evaluations frame our open-weight release decision. First, we aim not to release open weights that would extend the absolute frontier of cyber capability. K3.1 Pro's offensive cyber capability is, by design, below what our pipeline can produce, and remains behind that of frontier proprietary models. Second, we intend to progressively raise the cyber capability available in open weights. We believe making K3.1 Pro available in a non-discriminatory way benefits people who need dual-use capabilities for defensive purposes. When a capable model is open and runs on infrastructure its users control, many more teams gain the means to audit the code they maintain — including legacy systems that rarely receive dedicated security reviews. We believe that letting this capability diffuse gradually — rather than concentrating it in a few proprietary systems — ultimately makes all software more secure.

I don't think Chinese labs are behind at all. I think the tell is that they keep releasing models 2 weeks after new American proprietary models that match or beat them. Suggesting they're sitting on much higher capabilities and releasing just enough to undercut

Could be part of the reason that the frontier models are getting the administration to go a little harder against Chinese models.

> Which means that the US lead is all but gone.

You can see in this discussions the goalposts being actively moved to desperately create a notion that the lead is still there.

You even see people in this thread referring to imaginary wonder models that they expect US companies to have under development that are supposedly leaps and bounds more impressive, albeit the only reference there is to their existence is the commenters hopes and dreams.

I think people need to realize that there is no longer a cutting edge company that is expected to dominate, and among the multitude of AI companies in the field competition is so tight that no one would even be surprised if tomorrow some other company far away from the US was able to crank out the top performing model, or something like a small local model that outperforms mid/high tier frontier models. The local model alone would be a death sentence to US companies, who are still burning through 10x their revenue.

Muse Wonder is even higher, and there is no way in hell it's better in practice than either Astra or Fable.

> When evaluating the Intelligence Index, it generated 150M tokens, which is very verbose in comparison to the median of 87M.

So their base price is 1/5th of Opus 5.5 per token, but if you want "highspeed" it's 2x that and then I guess it generates 2x the number of tokens as well?

That might suggest that Opus is competitively priced after all.

> When evaluating the Intelligence Index, it generated 150M tokens, which is very verbose in comparison to the median of 87M.

This is 100M fewer than Opus 5.5, 40M fewer than Fable 5.1, and about a third of Sonnet 5.5. In the top-6, it's 3rd, beaten by Muse Wonder at 140M and Astra at, impressively, just 60M.

I don't trust these Artificial Analysis indexes and I'm not sure why anyone considers them authoritative, the site just sprung up a few months ago and now everyone quotes it?

From my own use of the models I find their ranking to be totally inaccurate.

while this is an awesome model, I have a bridge to sell to folks who think (based on AA leaderboard) that this is better than Opus 5.5

> beating both GPT-6 Astra and Fable 5.1. It's also faster than both.

Very similar to my adversarial programming language (KillSwitch) benchmark[1], at least on pass rate, it is much slower though.

1 - https://bench.killswitch-lang.org

From that link:

> reasonably priced when comparing to other models of similar price

???

Im running Qwen 3.8 Flash Next with an abliteration LoRA, its pretty good. I only really use claude sonnet outside of it, and it is on par

K3 and GLM 5.3 flash are pretty mid by comparison when you consider how much more it will take to run them

I mean... They just copy the US's homework, so I would hope they can run a leaner operation.

Talking point #6 - https://www.reddit.com/r/singularity/comments/1xdquz5/kimi_a...

This is a price drop from Kimi K3 ($3/$15/$0.30) despite being a slightly larger model (3T vs. 2.8T params).

I'm glad I waited a day to post my analysis of LLM price cuts because it needs...an addition.

Welp, that ruins this news for me. I'm glad open models are catching up but if I can't practically run them locally _or_ through a flat fee then it doesn't really help me. I fear API pricing.

Yeah, the plan is far too expensive for what it provides. I wonder if the quota with these new models will be better, but I'll wait for others to try this out before wasting another hundred bucks.

It also has the lowest quota of any model on https://opencode.ai/go

I think Kimi just doesn't have the means to subsidize plans.

Had a similar experience here. Their limits are very, very low, and ultimately make it not worth subscribing. Can't leverage a good modal if it exhausts its 5-hour budget in 30 minutes.

Paying API prices for Deepseek 4.1 Flash feels like a much better deal at the moment.

Ya it's not very usable compared to Claude / codex

Same experience. I had the $19 monthly plan for experimenting and running benchmarks, but it was a uselessly small amount of usage, a single task would blast through the whole five hour limit and a big chunk of the weekly limit. All of the US provider plans are remarkably more generous (though OpenAI plans have gotten a lot stingier lately, where they used to be the most generous).

What about via US hosted data center resellers like Fireworks?

Yup, insanely low 5-hour limits.

Honestly, use Pi, start the session with K3, design it, store the design + chat history + todo with phases + why + readme then when you are at the time to build do `/model` select deepseek and let it rip!!

We really need consumer hardware to catch up so we can run these models locally...

How? I've been running k3.1 Pro with K2.8 preview for codegen (subagents) and I've only used 22% of my weekly and it's been running for 8 hours on a RE project.

Are you using k3 with thinking set to the absolute highest and only using k3 for everything?

I have the same plan and have never experienced that with K3. I have hit the limit within a 5 hour quota using Swarm, however.

It seems that GLM-5.3 is still the go-to for cybersecurity.

Also the speed

Hardening in this case could mean biometrics, or it could mean rethinking the trust model so that controlling an army of duplicates confers no advantage because you have to be trusted in order to do any damage anyhow. Containing multitudes buys you nothing.

I hope we chose the latter. Biometrics are creepy.

Hardening against what? Apologies if I am being dense here.

The way I see it, automated systems should basicially have access to the financial system anyways. This is some of the appeal of cryptocurrency.

Sybil resistance built into the traditional financial system (for example firewalling based on residential IP, requiring banking apps, etc.) is a massive annoyance that can obviously by circumvented at scale.

I imagine that Chinese AI labs would be more than happy to pay for tokens to train on. If Anthropic goes and bans them as quickly as it can identify to them, of course they'll shift to using lots of cheaply obtained sockpuppets to keep getting the same data.

I doubt that an AI could do anything with these that legions of scammers haven't already figured out how to do - after all, it's not like AI set up this distillation setup in the first place :)

No offense but you don't know what you're talking about.

Spitballing random pr pieces from companies about to IPO is not helpful to anyone, fyi.

Color me naive but I can’t believe that approach works at any reasonable scale. Any meaningful token spend at distillation scale would be raising alarm bells at all the cc issuers.

Why do we believe all of this without a shred of actual proof. Breached computers and stolen cards is straight up crime, you'd think that there would be a lawsuit or two then.

Why is that a talking point now, all of a sudden?

From my POV it looks like you guys tried "They're distilling us! Very immoral!", saw that and nobody cares, and now you're trying to run the more emotionally impactful "They're distilling us... with your grandma's stolen credit card! They're stealing from your grandma!!"

Probably something like:

MiniMax -> Qwen -> DeepSeek (pro) -> Gemini (Carbon) -> Fable (to beat Carbon) -> Astra (to be slightly worse than Fable but cheaper).

I doubt we'll see any major US release besides Grok in the next 3 weeks.

Would be nice if we got a Terra that's competitive with Gemini Flash and DeepSeek Flash.

The fun part is, we're already at the point where we are seeing something frontier-ish almost every single week.

The really fun part will be when they all start adding System 1-3 natively (IMO) - which I think will be the next release cycle.

They don't want to be banned, and maybe their best offensive cyber capability models will be for their govt.

The more interesting question is how much more performance can be squeezed out by post training the open weights through RL.

Or it's a cop out. I could totally outrun Usain Bolt, but by design I don't push to set new World Records. I will increasingly show faster times though.

This doesn't seem particularly notable and is in line with Anthropic and OpenAI's approach of restricting release/access to more capable models.

Yeah that's a huge downside. That the model does the task without tripping over its own feet, like discovering a vulnerability the model itself accidentally built 15 minutes ago and triggering its own guardrails, made the Chinese models attractive to use for security-adjacent things. Also useful to red-team your own stuff. While I can understand the reasoning behind it (and I guess having a model that's really good at both offense and defense released into the ether could lead to CCP asking some very pointed questions if Moonshot managed to get blamed for some incident stemming from their model's misuse) it's still a huge disappointment.

I don't want to give US labs the results of my DNA tests and my family tree going back to the 16th century just to secure my shitty app, but if the Chinese are bowing out as well it seems the only alternative is to admit that some script kiddie with a bunch of tokens will just pwn you now and then.

On the surface that seems like a responsible thing to do

All it takes is one idiot who happens to have the resources to "pWm Teh Interwebs" to scare both the US and Chinese governments.

Kimi is the closest Chinese equivalent of Anthropic.

It's not actively nerfed like the Western frontier models, it's just downweighting offensive "cyber" as a domain for post-training/fine tuning. Which, if anything, ought to improve model capabilities wrt. other domains.

> So they admit that they could do more, but they won't?

Yeah, there's another paragraph:

> Identifying and removing cyber-related data. During development, we observed vulnerability research performance improving steadily as post-training scaled. Tracing the source of this trend, we found that cyber-related content was embedded within the general mid-training corpus, and cyber-related tasks were present in the RL environment mix — a natural byproduct of training on broad data. In preliminary experiments on internal models, we found that without data removal at either the mid-training or the RL stage, downstream cyber capability would increase substantially. These findings gave us a concrete lever for control. For the open-weight release of K3.1 Pro, we identified and removed cyber-related data from both training stages, while retaining the general-purpose coding tasks needed for model usefulness. We will continue to apply appropriate controls at every training stage, of which cyber-related data identification and restriction is one example.

(emphasis mine)

It's clear that cyber is a touchy subject right now. But it's also clear that RL works on it, because it's highly verifiable. So smaller models are going to become very interesting, because they can be fine-tuned later to cover this field.

  > It's clear that cyber is a touchy subject right now.

I mean they say they will release the weights by November 4, so presumably after that it won’t be hard for people to fine-tune it to be better at cyber right?

Moonshot (Kimi) seems to be unashamedly into American culture[0], I wonder if they're paying closer attention to and giving more credence to some cultural aspects of American AI discourse including aspects of AI caution/panic.

[0] https://x.com/crystalsssup/status/1944287779896328668

(note: extremely speculative and based on almost nothing, and especially near-complete ignorance to alternatives)

They will. Just not in the version they release to you.

Isn't this done so they can defuse Anthropic's excuse for regulatory capture?

With frontier competitors and the U.S. govt actively pushing "concerns" about Chinese models, Kimi including this statement on "offensive cyber capability" seems like smart defense. Models are opaque and training details are trade secrets, so whatever they did might be negligible or significant and no one can tell. This bit is also artfully done:

> "below what our pipeline can produce, and remains behind that of frontier proprietary models."

Like Ant and OAI, they've realized vaguely claiming to have nerfed their latest model to an unspecified degree not only looks 'good guy' prudent, it provides a built-in excuse when a model fails.

Oh great, more censorship. We wouldn’t want those pesky plebs securing their work.

Read this as ‘dear government we can train a cyberweapon if you pay us’ assuming they wouldn’t say anything if they were already paid to do that

We really need some breakthrough like DiLoCo does, but for heterogeneous unstable consumer hardware to make volunteer crowdsourced AI training a reality. Yeah, I understand from current technical angle how much of a mountain that is. Call it AI@Home or whatever. I never thought it would bother me personally or even become a problem, but these companies acting like gatekeepers and virtue signaling while also in bed with governments etc.. this leads nowhere good.

Cowards.

The way to stop the vulnerability apocalypse is to fix the vulnerabilities, not to shoot the damn messengers.

I hope someone abliterates and FTs this model on vulnerability discovery so we can find more bugs in our testing, insecure tech stacks.

Good point. Wouldn't be good if there were white hat models being developed by legitimate organizations at this point?

I hope one day they learn to inhabit their position more confidently instead of merely aping their competitors.

Haven't gotten to try it yet but that'd be a breath of fresh air. Too used to LLMs flip flopping at the slightest hint of challenge.

I always appreciate when a model does that. It's amusing, but also having it be argumentatively wrong is a lot less annoying than being sycophantically wrong.

It could be the system prompt

Eh, there's good and bad. The model mostly sticks to prevailing narratives in the training corpus. So if there is something that the prevailing narrative is really wrong about, it's gonna be just as pig-headed about it as a typical internet forum rando.

Reminds me of early test versions of Gemini Flash 2.0 (or was it 1.5?). It would argue something, try to prove itself right, realize it’s proof is agreeing with the user halfway through and, instead of admitting defeat, would backtrack to try another approach. So much fun.

I much prefer that for chat, but I'm somewhat worried that we shouldn't give tool-enabled agents an attitude. It's all roleplay, and I wouldn't want a system playing the role of an annoyed assistant to misuse its tools.

I might want to try that. DS 4.1 Flash anyway fares much better at it than GLM (which is quite not so good at this aspect). Hope Kimi is even better.

Eh, not necessarily, many of the RL vendors sell to Chinese labs, too: https://www.interconnects.ai/p/glm-53-how-chinese-labs-keep-...

They're cope, as if Anthropic or OpenAI have some magic moat, and not just basically a headstart and more money.

When people say distilling they mean SFT on other models outputs.

In their thinking and if you ask them, I've seen Kimi K3 and other models confidently say that they are Claude from Anthropic, and accuse you of attempting to jailbreak them if you say otherwise.

Maybe it's intentional so government doesn't block open weight release.

There was law proposal (rumour?) last week about Chinese labs having to get approved for weights release now.

You either include all labs which can reasonably produce models which are capable enough to be considered generally on-par with the best available models, in which case there has been way more than 2 frontier labs for a while now, or you take a more literal definition and only include labs which are capable of producing models which are considered "the best" out there, in which case there are basically just 2.

I think the former definition is a bit more meaningful since it avoids issues with determining what actually constitutes "the best" model out there. In that case, Google has been a frontier lab for a long time, Meta arguably as well, and you're also ignoring xAI which has been competitive for a while now. And, obviously, there's a bunch of Chinese labs which would deserve the title as well.

But if one is being strict (and willing to accept generally accepted definitions/measures/etc.), then there has only ever been two frontier labs: OpenAI and Anthropic. They're the only labs to have had "the best" models at any point in time since LLMs became a thing.

Moonshot has been frontier since this summer. At this point whenever they release a model it's on par with whatever the latest OpenAI/Anthropic model release was.

Not sure I'd put Google there in practice. And while Muse is an impressive jump from behind it's still not quite at the frontier either IMO.

Muse Wonder is good but it's not frontier. It goes neck and neck with Opus 5.

I'm not sure what counts anymore. You've got OpenAI and Anthropic; then Meta, Google, and X; then over to Alibaba/Qwen, Z/GLM, Moonshot/Kimi, Deepseek; and Mistral in Europe. How many people out there in the real world can actually come up with tasks that differentiate those 10?

right now it looks like only anthropic since openai made the staggering business decision to release their midrange model (terra) rebranded as sol 6.1 and quantized astra into the ground.

everyone else mostly seems to have just decided to cheat on benchmarks to make their models look better than they actually are, and none seem to want to actually try to make better models.

It's still just 2. Google hasn't been at the top in a long while, and even when it was, it was just a tiny improvement over what other labs had previously released. Meta has never been at the top. Only two labs release LLMs that really push the limits out.

> How many frontier labs are there right now?

As long as LLM's can be distilled, and all these "labs" (what a weird name) can distill one another, there are going to be as many as capital is available to be burnt on the topic.

About how much would the session have cost if you were paying API prices (assuming good cache rates)?