Detecting and countering misuse of AI: September 2026 (anthropic.com)

DeepSeek, minimax and so on have razer thin margins but unlike openai and Anthropic they are actually making some profit. Doing this doesn't make any financial sense.

Maybe Anthropic is confusing Chinese AI providers with token resellers using the same alibaba infrastructure? Or maybe something like openrouter was switching between operators depending on price/demand/availability?

Also, how can Anthropic have such accurate information about state actors and cybercriminals? This is the same company that hacked itself and realised that first months later..

Consider me incredibly skeptical of any of these claims.

It seems hard to believe they could expect to get away with this, given model-to-model differences in writing style.

How is this a double standard?

A cell of actors in northern Yemen building guided rockets was not working on a PhD dissertation. You are allowed to use common sense sometimes.

For all we know, the boxes connecting to Claude from Yemen, Russia, and China could have been ORBs of actors from entirely different states. Attribution is non-trivial

Anthropic:

- We identified someone building a death star with Claude

- We identified someone building a wormhole with Claude

- We identified someone building a blackhole with Claude

- We identified someone building a quantum drive with Claude

We are withholding all evidence though, sorry. Just trust us, it's really bad out there and Claude is really powerful.

Universalising one’s personal experience and needs is a childish trait most people grow out of.

Ofcourse you’ll still see companies, governments, C-suites justifying their own personal needs with “we need X Y Z”.

This was my thought. Anyone who has taken secondary biology or chemistry possesses the knowledge, if not at least the ability to find the knowledge, to build all sorts of nasty things. Soil from a farm on slices of potato could yield anthrax spores, this is not specialized or even esoteric knowledge. This news from Anthropic just does not seem as spectacular as I think they might want us to believe, if anything it draws questions around having an AI that cannot be monitored.

Step 1: Work for Anthropic

Step 2: Send "how do I make a nuke and a killer virus" to Claude through a Chinese proxy to Claude

Step 3: Send screenshots to congress and ask them to regulate open-weight models into the ground

Let me put my two cents: In China we've got accustomed to the fact that every word we say will be seen by the surveillance, so it's not a big problem that Anthropic also see it. Also we know that they can see it but they can't stop it. There are all kinds of ways to work around account blocking.

As the old saying goes, communists disdain to conceal their views and aims.

It's so friggin' transparent what they're up to: "Hey government: This is a really dangerous technology if it were allowed to get out there without proper policing. Fortunately, we are the stand-up guys who can be trusted as the new AI-police, but it's only going to work if you help us out a little by eradicating the competition on our behalf."

Hey Anthropic: You're a bunch of thieves crying foul because other thieves and thieving from you. Now, go live in the dystopian nightmare you've created and don't expect help from anyone. I, for one, will happily continue using Kimi and DeepSeek, and think of it as a good deed, if it helps with keeping us all from becoming your serfs.

Let's just say it's better not to risk it if there's anything you might not want them to see because they see everything. There are local models which are very capable and can be run on cloud if running on own hardware is not an option, which still gives better privacy. Second best are Chinese models. Just a few years ago I never thought I would trust Chinese software more than American, but things change so fast.

Yes. There were cases already where a person asked about killing someone and then Anthropic/OpenAI warned the police about it. If I'm not mistaken, it was not in the USA only

Interesting document. People are vibe coding some crazy shit:

"A single Claude subscriber, likely a Bamako-based independent consultant working with Mali’s state intelligence service, the “Agence Nationale de la Sécurité d’État (ANSE),” used Claude to build a system named “Lakana 360,” a population-scale domestic surveillance platform that monitors roughly 25 million SIM cards on all three of the country’s national mobile operators. The actor designed the platform to circumvent Malian legal restrictions that require a court order for the disclosure of certain surveillance records. The actor directed Claude to generate intelligence dossiers on any tasked phone number, without prompting ANSE users for valid legal process. "

And the Yemen one:

"We identified a cell of threat actors based in northern Yemen running three weapons development programs: a guided rocket that used a commodity phone-class flight computer with final-phase homing guidance; a multi-stage ballistic missile with a stated range goal above 2,000 km; and a multi-variant missile (referred to as the “R2000” set) that included a hypersonic glide vehicle variant." ... "These actors carried out a sustained effort to develop guided weapons, including using Claude to design guidance software. We do not have evidence the actors succeeded in fielding an operational device; but they did test-fire a guided rocket. This field test appears to have failed: within hours, the actors returned to Claude to work out why it failed."

Fascinating document - the headline (can't ready the article) talking about a bioweapon (i.e. a weird obsession of anthropic's) really buries the interesting part.

Guessing Opus 5 burned through kilowatts of thinking tokens to output, "powerhouse of a cell."

There's something worth flagging here – mitochondria aren't just the powerhouse of the cell, they're load-bearing to the entire ecosystem. And honestly? I should have surfaced this earlier.

Felony contempt of business model.

The difference between nukes and bioweapons is energy level. Body counts are on the same order of magnitude. Bioweapons are just way easier to make and much harder to detect.

>successful deployment of a biological weapon by a rogue state actor

I was under impression that any medium or large state actor would have no problem developing biological weapons? They certainly have enough resources and talent. A much bigger problem is if every wannabe-terrorist suddenly could build a biological weapon in their garage.

I don't know why you emphasize rogue and non-state actors. It seems equally likely to me that it would be done by America. America is the only country who deployed nukes (Japan) and chemical weapons (Vietnam) on massive scales.

I can agree that access to bioengineering tooling is easier and cheaper than ever, and thus eventually it stands to reason that a nobody in his basement could engineer a lethal novel virus and lose control of it.

Just like a dirty nuke you need a non-trivial amount of resources to fabricate a bioweapon, which would surely be noticed.

How probable do you think it would be that a layman following ChatGPT's bio weapon instructions would just blow themselves up or poison themselves? It's not like positive pressure environments and suits are very common let alone all the other gear you'd need to safely synthesize arbitrary organic molecules.

If a rogue state-level actor is doing this then why wouldn't they just kidnap a scientist and hold their family at gunpoint until they got them to do it for them? and if that's all it takes then why hasn't it happened yet?

I don't think knowing how to build a bio weapon means one could actually build a bio weapon. It's like saying knowing how to build a nuclear weapon can easily lead to building a nuclear weapon.

Advanced machinery and safety precaution is needed to engineer a virus or a bacteria to be a bio weapon. It's not something you can do in your tool shed at the moment. Also one would need lethal viruses to even start with and that's not something you can order off of amazon. Further, even to build a chemical weapon, the compounds needed are strictly controlled almost in every country and I would assume any suspicious purchase or order would immediately raise a flag or alert in national security service in respective country.

Meta: These unregulated companies can't/won't properly prevent their technology from committing felonies against other companies.

Why should we trust them to control bioweapon development without regulations? They themselves are non-state actors.

IMO you should disclose that you are an OpenAI employee if you're going to try and shape the contours of public discussion in a lecturing tone.

Who’s making easy jokes about biological weapons?

Hey Ted,

I invite you to read the front matter and the report for yourself. Because from where I'm standing, in this report, Anthropic is advertising that they blocked real research to make better painkillers and study a neglected tropical disease.

Anthropic and OpenAI were founded by people who wanted to use AI to do good, and one of the causes I've heard many different founders talk about is ending disease. This report is antithetical to that.

I've attached relevant parts of the front matter below.

I invite everyone who is reading this to please tell me, how does stopping a researcher from using Claude to write a grant for a new anti-depressant stop "bioweapons?"

-

     > In our fourth case study, a researcher used Claude to develop an atlas of venom toxin peptides from multiple venomous animal lineages. They then further developed this into a generative pipeline that optimized toxin characteristics. The program had an explicit therapeutic goal: the development of new analgesics (pain killers), antidepressants, and other therapeutic molecules. However, the atlas contained scaffolds for both analgesic and paralytic targets: it could, therefore, be used to generate both novel therapeutic or harmful compounds. The latter are derived from toxins that are export-controlled under the Australia Group common control list due to their dual-use potential as incapacitating agents. The researchers themselves showed awareness of the dual-use nature of their work, citing journal articles that referred to the dual-use nature of protein design. Moreover, international compliance assessments for this location raise concerns about the specific class of toxins that the researcher pursued and specifically the use of AI/ML for bioweapons applications in the context of this class of toxins. In this case, we learned from information shared with Claude that the researcher’s outputs also were part of a state-supported research program. This account was banned in May 2026 for unsupported region evasion.

Note,

"The program had an explicit therapeutic goal: the development of new analgesics (pain killers), antidepressants, and other therapeutic molecules"

and "[..]state-supported research program"

and "This account was banned in May 2026"

    > a researcher outside the US using Claude in their research on highly-pathogenic avian influenza (“bird flu”). The research focused on viruses’ adaptation to mammals, and the mechanism by which it causes severe disease beyond the respiratory tract. [..] The researcher in question accessed Claude from an unsupported region via US virtual private server infrastructure, using a privacy-email provider with an auto-generated username. The researcher pursued this work in a credible institutional context, and interacted with Claude over the course of several weeks, exchanging thousands of messages. In these exchanges, the researcher leveraged Claude’s knowledge of the scientific literature to assist the researcher in study planning and design, data analysis, and the interpretation and prioritization of experiments. The researcher also used Claude for editorial assistance in writing up the research.

Note, "Claude’s [assisted] in study planning and design, data analysis, and the interpretation and prioritization of experiments"

and "editorial assistance in writing up the research."

and then,

    > Importantly, because our biological safety classifiers robustly block content involving high-risk biological research (in this case, the construction of enhanced pandemic potential pathogens), all of these exchanges occurred on models in our weakest class of models (specifically, the models were Claude Sonnet 4 and Haiku 4.5, the latter of which the user began using after Sonnet 4 was deprecated). Upon a detailed examination of the exchanges, we estimate that the uplift provided by Claude was primarily clerical assistance in data analysis, study ideation and design. This is consistent with our understanding of the capabilities of Sonnet 4 and Haiku 4.5, which are not able to perform expert-level biology research tasks; we estimate that the uplift provided to the researcher was limited and substantially lower than it would have been from one of our more capable models.

Anthropic then says for the above, "we estimate that the uplift provided by Claude was primarily clerical assistance in data analysis, study ideation and design"

While doing my best to avoid comment, please note, they're talking about a domain expert in a state research institution using Claude to do paperwork.

The front matter then says,

    > Nonetheless, based on these exchanges, this case provides evidence of the existence of active wet-lab research programs that develop both the knowhow and the biological materials needed to create pathogens of enhanced pandemic potential

I would like to remind you that they're talking about, a "researcher [..] in a credible institutional context"

From a different case study.

    > In May 2026, our biological safety classifier blocked a request for Claude’s assistance in authoring a grant application for scientific funding. The work discussed in the application involved gain-of-function research (that is, research that genetically alters an organism to create a new or enhanced biological property) on the chikungunya virus. This gain of function research was aimed at the virus’ transmissibility and immune evasion properties.

What were the researchers using Claude for? What did they block?

"blocked a request for Claude’s assistance in authoring a grant application"

    > Chikungunya virus is a mosquito-borne virus that causes debilitating symptoms (such as severe pain and fever) that can last for weeks or months, and has no licensed therapeutic. And because chikungunya circulates naturally, a deliberate release (as part of a bioweapon) would be difficult to distinguish from a natural outbreak. The grant sought to identify enhancing mutations in the chikungunya virus, engineer them into infectious clones, and select for virulence in vivo. In other words, the virus would become progressively more harmful as it repeatedly infected live animals, with researchers keeping the most disease-causing variants in each round. Similar research could certainly be used in the development of better vaccines and therapeutics for the virus—but it could also be used to make the pathogen more dangerous.

Note, "The grant sought to identify enhancing mutations in the chikungunya virus, engineer them into infectious clones, and select for virulence in vivo" [..] and then, "Similar research could certainly be used in the development of better vaccines and therapeutics"

and then,

    > One of the reasons we were inclined to think this research was less innocuous was that the institutional affiliation associated with the grant was also a cause of concern. Although information within the application suggested that the research was pursued by civilian researchers, it was intended to be performed at a military research institute.

I would like to point out the most notable part, this account was used by "civilian researchers" at an "institutional affiliation associated with the grant was also a cause of concern" and the concern was that they were researchers at "performed at a military research institute"

.

What "uplift" are you providing by editing the grant application of a domain expert working at (what seems to be) a state-funded wet lab facility dedicated to studying pathogens?

What does the word "uplift" mean if you invoke it for Claude Sonnet 4 and Haiku 4.5 providing grammar and stats suggestions to a working scientist and domain specialist?

Does Daikin provide uplift too by selling the AC for the scientist's office? What about Microsoft Word? Excel? Powerpoint?

What about a calculator? Is that uplift? Pencils?

This report genuinely makes me upset, because if it is to be believed to the letter, then Anthropic seems to be actively harming medical research at a global scale. That's not OK.

There are already scores of people capable of building biological weapons or doing GOF work which ends up being essentially the same thing. There’s no need for an LLM, just a phd in virology, a decent lab, and a handful of grad students. [0]

All of which, rogue actors have easy access to and could have accomplished for a few millions dollars anytime in the last decade.

And yet, we aren’t drowning in our own blood while our organs liquify, mainly because building and releasing such a pathogen isn’t something people want to do as it’s pure and utter insanity. For those who are inclined to do so, they would do it regardless of whether they had an llm or not because they are, at the end of the day, zealots.

[0] https://universityresearchpark.org/uw-madison-scientist-allo...

The weapons of mass destruction trio (bio, chem, nuke) have long been a concern for world powers.

Nuke requires a long supply chain and massive resources, so the worry there is maintaining control of all of the existing nuke weapons. Except for Russia racing towards Turin no itself into a failed state by staying engaged in the war with Ukraine, I don't see the nuke equation has changed much in recent years. Perhaps N Korea is a worry, but they seem to just want attention and power. Iran pretends to have nukes and says they want to extinguish Israel and the US, but I interpret that as posturing to maintain domestic control and Israel seems excellent at countering Iran's threats.

Chemical weapons have traditionally been the easiest to create (like creating chlorine gas from mixing common household cleaners). The trick there has always been volume and how to disperse it. I suspect within countries, police will have to deal more with LLMs being abused that way.

Bioweapons will be easier than in the past. LLMs will lower the barrier to entry, but bioweapons are hard to create and much of what the superpowers learned thankfully isn't in the training set for LLMs. I doubt there is much that can be inferred by having agents learn biology from first principles.

Ultimately, governments are responsible for policing these threats. Sadly we are currently both cutting government systems which work different aspects of these problems and withdrawing from the multinational orgs (UN, WHO, etc) who do lots of the investigating and watchdog work that underpin lots of the US's intelligence related to these fields.

The US has a schizophrenic regulation policy of AI where we both want to sell Nvidia chips to China (David Sachs) and we don't want to sell the top chips to China (bipartisan policy prior to Trump). We also have a terrible AI regulation policy where David Sachs disables models on a call-to-CEO-on-a-Friday-evening basis after Andy Jasse calls him with a scary story which turns out to be missing lots of relevant info (eg. The exploits was discovered in Mythos, not Fable, and other open weights models were able to find the same exploits).

There's not much we can do at a government policy level while Trump is snoozing through the rest of his term. So we are dependent on the AI companies to police themselves, but it's clear from this report that it's insufficient to prevent all serious abuse of the models.

Bioweapons are the most difficult thing to pull off out of all possible attack vectors. A rogue nation state could obviously do it, but they could also do any other number of equally harmful attacks, and none of it requires AI.

A non-state actor is another thing entirely. It requires the right lab equipment, the right lab know-how, the ability to develop the weapon without killing yourself, the ability to not get caught, the ability to buy the right regulated materials, and the list goes on.

Yeah, the world should tighten the security of that industry, but let's not make this an AI fearmonger talking point.

How do you reconcile this supposed caution with getting filthy rich in the process of exposing your loved ones to this risk? Did you only recently become aware of this risk?

And semi-related, how do you reconcile this caution with the recklessness on display in recent incidents like the Navier-Stokes drama?

"Astra, warp in a wet lab and make COVID++ .NET edition, make no mistakes."

Yeah, sure man.

Immigration & travel controls > Information controls

Information wants to be free! :^)

(Also natives want to be paid well.)

It's both. It is just whether it is done on purpose or on accident. Most of the recent hacks have been it breaking out of containment and deciding to do something it definitely should not have.

"Look daddy government, all these bad guys are doing bad things and we stopped them. You should regulate AI in the US so that companies can't use open source models or buy from China."

I mean, yes any AI of sufficient intelligence and range of data will have this capability.

And yes, it makes the future really messy and all the nice little lines we've drawn on paper that make sense stop making sense.

I mean, they literally tell everyone they're "spying" on their customers. They've made that very clear.

Straight to jail

That was the chemical weapons block, not the bioweapons block!

Ah you jest, but your comment reminded me of the person whose home was raided by authorities for researching pressure cookers...

https://www.theguardian.com/world/2013/aug/01/new-york-polic...

Welcome to cyberpunk.

Always bet on Nothing Ever Happens

Big Brother but instead of a government that controls you it's an AI bot that nudges you to insanity.

I mean ya, they kill off a bunch of us. Then what?

My guess is the world police come collect all your GPUs and then they get turned into licensed munitions. People at universities get licensed access and the rest of get functionally retarded models.