Dario, Please (pop.rdi.sh)

It’s not lobotomized, it’s a simple harness restriction that has nothing to do with the model or its capabilities. And either way, I’m not sure what that has to do with “negligence” or “intent”? You think frontier labs should be prosecuted because they don’t allow agents to turn off your PC?

Anthropic and OpenAI are both behind Cloudflare. It's fairly easy for an upstream to shut you off. Beyond that, the government / law enforcement could seize and disable their DNS within an hour.

This seems highly unlikely to be a problem. Most of the interesting/dangerous models are too big to fit in a single GPU instance. Once you have to spread across "normal" networking, performance will be crippled. Then there's the problem of billing...

> Agents could make a virus that does not require continued inference to do it's thing.

Sure, then it hits a poorly-designed part of its code and effectively dies. Without an experienced human in the loop, I have my doubts as to its practical severity.

> Agents could take over the internet in a way that isn't immediately detected by those companies, so that by the time they do shut off API access the damage is done.

Billing is a likely limiting factor here.

> OpenAI or Anthropic could choose to not shut off API access, because the hack is bringing them in money or furthering their political aims.

This is where citizens with access to backhoes come in.

> Agents could also hack Anthropic/OpenAI and make it appear that API access has been turned off, when in reality it hasn't.

Billing and other usage metrics would be an obvious tell.

You know cables, modems, RF equipment and optical transducers can all be unplugged right?

What defines a model? What defines ownership of a process? If I make a wrapper to a remote VM that builds and executed a prompt, am I accountable?

When I worked at a company in the EU, it was enough to apply a reversible linear transform to the data for it to be considered GDPR safe-according according to legal definition as long as the transform details were stored separately.

"A series of tubes" was the same kind of political character assassination that led to Howard Dean getting ridiculed for his infamous scream. He butchered the sentence. Fair. But Stevens should be ridiculed for parroting a tech industry lobby stance about net neutrality, not for the series of tubes metaphor.

You are probably too young to remember that the dominant metaphor for the internet in 1990s politics was "the information superhighway." It was easy to think of the web as "driving" browsers to visit web "sites", with slow bandwidth being analogous to being caught in traffic. But the internet is closer to water, gas, and electricity than roads. Concepts like bandwidth and throughput are closer to how they play out in infrastructure policy for various things with tubes, versus cars and roads. Do you think he's wrong and that the internet is closer to "a big truck" versus "a series of tubes"?

The issue being debated was net neutrality and bandwidth, including specifics about who pays for what and the downstream second-order consequences of various policies. He was parroting some line from some telecom lobbyist, but the point the lobbyist was trying to make through Stevens was about how if certain policies about who pays for bandwidth were adopted, it could disincentivize some things at the Tier 1/2 layer that could increase transport costs at the Tier 2/3 layer that impacts ordinary people's bandwidth.

What if a person physically broke into the car and did the same thing? Clearly they are the one to blame then.

The whole person in the loop is liable is already an outdated concept when decisions are made beyond the persons physical control.

At this point, in the US at least, the laws give the corrupt elite the ability to deter competition and to target the people who try to get in their way.

In other words, it's protection for the potentate and his sycophants, not the plebs.

> If we passed a law that said the CEO of any company that deploys an LLM that commits a crime gets punished as if they personally did the crime (so, basically instant life sentence if it's even a simple crime times a million instances), I guarantee you the first email the CEO sends to the company is a "pause every LLM project we have - we gotta think about this".

Why do no such laws exist in practice? Why are corporate crimes almost always settled by payment, not individual punishment?

If you answer these questions, you'll know why the law you envision has a lower chance of being enacted than AI destroying humanity.

There are laws, but if you’re rich enough, the laws don’t apply.

Boeing was responsible for the deaths of hundreds of people. The people that facilitated this weren’t held responsible and were in fact compensated to the tune of 10’s of millions of dollars for doing their jobs terribly.

Laws are the last line of defense. People don’t do bad things primarily because their human nature and moral compass stops them from doing so.

Because there are many important values of "something" where such punishment acts as a deterrent to other would-be criminals; and because society is more stable when people see justice done and that mollifies hoi polloi after the damage occurs.

But many AI doom scenarios don't fit that paradigm. The first time "something" happens (at least if you buy the argument) would be bad enough for legal punishment not to matter.

It's just like you said: this only works if the threat of punishment is credible.

which the government doesnt want since this will stop progress, while other countries will continue to develop LLMs

It is also a choice to not do any or all of the above.

Not surprised this is always what they have and hack.

Who would use an Agent that spends $10,000 re-implementing some OAuth lib or reverse-engineering a proprietary lib when it's free on the internet?

I don’t agree with them, but I don’t think it was the raw compute power as much as it was maintaining the _delta_ in compute power.

(1) Give the big AI incumbents an anti-trust exemption so the they can coordinate without it being an illegal agreement not to compete, and

(2) Adopt mandatory supervision (by giving priveleged access to monitors) to the shared safety protocols of the big labs, by a nonprofit funded by the big labs, of everyone training, distributing, or hosting models.

(3) Adopt a policy of seeking international agreements to extend substantially the same rules to foreign actors training, distributing, and hosting models.

The part they are doing voluntarily isn't to promote the lobbying effort for these mandates isn't slowing down, it is giving their pet nonprofit the access for “independwht supervision” of their own operations to their own existing safety rules.

Closing the Barndoor after the livestock have escaped

It worked.

There are direct quotes from Nixon and Reagan that claims this to be true. That is enough evidence for me.

Akshually there is, and WW3 has been going on since 2010. (Mostly in places that aren't Europe.)

I know you think you explained it but you didn't. You explained how an LLM might become misaligned and hide it but for the LLMs that are not, why would they not be capable of detecting that something harmful is happening and defending against the misaligned LLMs actions? After all, it was LLMs that defended hugging face.

You really buy that ridiculous argument that "the AI broke out of its sandbox! We had no idea! It's dangerous I tell you, dangerous! Unless you make us the sole gatekeepers of this incredibly dangerous 'intelligence', everybody gonna die!"

Please.

Bring the CFAA[0] hammer down on these guys and watch how fast those "uncontrollable" LLMs get controlled. "Oh, gee! Prison? We can't control this stuff...but it'll never happen again!"

That whole thing was, and is, a bunch of hooey. Those running the LLMs are entirely responsible for them, there is no such thing as agency for algorithms. Full stop.

[0] https://en.wikipedia.org/wiki/Computer_Fraud_and_Abuse_Act

Anthropic needs regulation in order to prevent AI from becoming a commodity. This of course does not benefit all tech businesses equally, especially those that are not currently at the AI frontier. So when JD Vance talks about AI, he talks using the mouth of Peter Thiel who may not see benefit from the same policy as Altman or Amodei.

The rest is just public support posturing and most of that is bullshit meant to distract from the high rollers game of winners and losers. The philosophy is money and power, who gets it and who doesn't. Us normies aren't really participants in the game, except where we are being manipulated into cheering for one side or another, and with little stake in the outcomes (although selfishly, I'd be pissed if I didn't have open models to tinker with).

Open source is fundamentally a vehicle for commoditization. This is great if your business is not AI and your business is instead something like GPU hardware or some product that uses AI. But it means that eventually, selling AI is not going to be the money maker.

OSI proliferated open source on a business strategy called "commoditizing your complements". These big companies don't do it out of benevolence. It was pitched to them in a way that FSF did not (which was more about morals and ethics, something business care little about), and it caught on. And the software business became about ads, consulting and cloud services instead.

Also, Democratic Republic of {Congo, Korea}.

And this is actually, scarily, true...

First sentence: "This incident has Dario shook, there ain’t no such thing as halfway crooks."

How can you possibly believe this? What else would someone in his position say? No billionaire has humanity's interest at heart, so they try to hide that fact by building libraries (Carnegie) or creating foundations named after themselves.

All of his words and none of his actions support his caring at all about humanity, and you choose to believe his words? The man has an IPO coming up (probably one of the biggest ever!), he could not be in a less trustworthy position.