Progress on provable inference for large language models (anthropic.com)

> VerInf proved Llama 4 Maverick (the same 400B-parameter MoE model we prove) at ≈0.02 tok/s.

Live by the sword, die by the sword...

You can also use this and create explanation video on it on Youtube, post on X, Bluesky and other in-detail stuff that only you and very little people are in the position to do.

I’m sure some are interested, and this could lead to a hire, too.

I know it’s always easy to say and I probably have less adversity faced as most people, but I really try to see them as energy enablers. Of the almost infinite options you have in life there is a big chunk of realities that are better than your current ones an adverse event can trigger emotions and therefore energy and push one to do things one wouldn’t have done in comfort.

Thanks for your work!

Is it hard? Seems like it isn't that hard if you use NVIDIA's remote attestation/enclave support instead of trying to do everything with ZKPs which have always been something of a dead end. CC overhead 2%, ZKP 100x+ and gigabyte sized proofs - just for inference. It's never been competitive.

I'm very skeptical anyone would care about this for training though. I mean, if there's someone out there who genuinely really really wants proofs of the dataset used to train model weights, let's talk? But it seems the value is mostly for open weights inference providers. They can prove what weights they're serving and they can prove they can't see people's prompts. It would take a lot of the risk out of using a no-name LLM cloud service.

Unfortunately, after many years of this stuff I have become old and cynical. The stated/revealed preference mismatch is huge. The majority of people say they want privacy in theory but in practice won't trade anything off to get it, and especially not convenience.

> The verifier runs on the client's CPU. It needs no GPU and only has access to a commitment to the weights. The client downloads the commitment, ~6.5 MB in all, and then one proof per request. Checking a proof takes 17 seconds on 24 threads (and 21 seconds on a single thread), including the offline phase's checks.

As a client that sounds like it might be much cheaper to verify than reprocessing.

> The integer version is generally expected and empirically tested to be within some small error rate of true floating-point inference generically (even taking into account non-determinism of floating-point computations), though this does not cover adversarially designed prompts.

[1] https://github.com/anthropics/FlashProve/blob/main/docs/what...

That's literally the most common gripe I see from Anthropic users and, so far from being "table stakes" there no existing mechanism by which those users could verify it is or isn't happening. If Anthropic can figure this out then most of this "is it just me or is Opus hurr durr today" noise can evaporate.

It's worth reading past the opening paragraph before asking these kinds of questions:

>When a user sends a request to an AI model, today they have no way of knowing for certain that the model they selected is the one that generated the response. For everyday uses, in which users mostly care about the output and not necessarily which specific model produced it, this is a less pressing concern. But there are several cases where such assurance would be useful. For example, independent evaluators who test AI models before and after release need to know that the model they are evaluating is the model being served. Clients who use a model through a third-party routing service want assurance that the tokens came from the model they paid for. And if a cyberattacker modifies the token stream while pretending to be a given model, a user has no way to detect it from the output alone.

This is from the first paragraph in the Introduction.

1. it proves your models are running correctly. FP ops tend to deviate in small amounts between hardware providers and tensor ops even more so due to the relatively new and unstable standardisation environment atm. So it proves your HW configuration is delivering outputs correctly. i.e. it's a good test for what is currently a very non-trivial validation environment.

2. it proves your external provider is running the inference they said they are. i.e. it eliminates the concerns of model providers running quantized versions silently behind the scenes since quantization breaks the proof and you need to use a new, updated proof signature for the quantized version.

3. likewise for fine tuned or tampered models. Or outright substitution for another model (ex the accusation by OAI and Anthropic that Kimi and DS and co are periodically forwarding requests to their platforms to harvest responses for knowledge distillation).

All in all it's a nice chain of trust utility if it can evolve out of the toy stage and reach production viability (since rn it's limited to int-only weights and 1k context which is practically useless atm).

This solves for the latter with a bunch of handwaving for the former. The end goal is gonna be FUD for cheaper model routing because that obviously is at odds with their biz model.

It might also provide an additional guarantee for users of Open Router and Fireworks.

It makes the inference more expensive and helps controlling the users.

> This moves unobservability layer to the system that collects the proofs.

I'd say it's adding security through a secondary system, rather than displacing security to a secondary system. Security by layers.

Watermarking checks whether a _specific text_ came from a provider.

PI checks that the text (that you already know came from the provider) was produced with _a specific set of weights_.

PI is mainly useful for AI supply-chain security.

- it only gives a high probability that the text was outputted from the model, whereas a crytographic proof is computationally infeasible to forge

- it cannot be traced back to the input. You could "wash" the output from one model into another and it would be watermarked by the latter. With the crytographic proof, if you have the input, output, and "commitment", you can verify your exact input was processed by the model (or so the model provider says).

In part because many of the accelerators they use may not support attestation, and even with ones that do, such as NVIDIA Blackwell, support is restricted to a single node (8 GPUs). Vera Rubin supports rack-scale encryption, but that is not broadly deployed yet and will not be the only accelerator they use to serve inference.