Cloud Agents Are Inevitable AI Prisons (normanponte.io)

Personification of AI is what’s going to get us in the end.

I think we need to draw a hard line in the sand over this. An AI didn’t hack into a company, the engineer set an automated tool to. An AI didn’t make an egregious security mistake, the engineer did.

We can’t blame the chisel for messing up our sculptures, when where just throwing the hammer!

There's a number of science fiction scenarios where the public internet becomes so vile a place that it simply becomes unsafe to be there.

The problem is that, in general, if you can get a bit from here to there, then you're going to be vulnerable to the possibilities of malicious communication. But we're going to want our AI agents to be able to get from here to there for a lot of "there"s; what's the value of an agent that can't speak to anyone? Much, much less than one locked away in a prison.

There isn't going to be a solution where we just lock them away and we just try really, really hard to filter everything they're doing. They're too smart for that already and we only want them smarter.

Basically, the security apocalypse we've been worried about for so long is upon us, albeit only beginning. Either we secure ourselves and all our services properly to the point that it's OK that potentially misaligned non-human agents are running around on the public internet and they still can't hurt us through our security, or the public internet becomes so dangerous that the only practical solution is to no longer connect to it and we all have to become very, very careful what we let through, to a degree of detail far beyond any current-day available network filter.

> I think this becomes the default. Give an agent a goal, let it work in its own environment, and come back to a result and a visualization of what happened.

I don't think this should be the default. There are many scenarios where we want agents to genuinely collaborate with each other. I have my Claude sessions coordinate work with each other, and sometimes with others' sessions over email or something. The idea that agents do the work, write HANDOFFs,and humans then act as carrier pigeons of said handoffs, does not really seem scalable to me.

>Knowing how a system does its work is how I’ve always made it better. You watch the process, you see where it wastes effort or takes the wrong turn, you fix that

The same thing goes with LLMs, on Codex, I just watch the process of the agent writing code, and if I see any inefficiencies or errors, I suggest a correction/idea, then Codex accepts/rejects and implement it; If there is anything about the code the agent wrote that I don't understand, I ask them to explain it to me so I can understand it.

It's not a complicated process.

Most protections you need for an agent are basic permissions capabilities of unix. Most risks of dependencies on cloud services are solved by not using cloud services, or using them only for things you can't in-house and choosing ones you trust a la carte. The paradigm of trusting some company with all your important stuff by default is naive and no one I know likes it, and it's more feasible than ever to run your own infra with tiny models smoothing out the wrinkles, and this is only becoming more accessible. I am working to make this true even for laypeople I know. Once broken trust is very hard to earn back, and many people's trust has been broken for years, they just felt like they had no alternative. As alternatives become easier and easier, I think people will defect

I think there might be also another possible way to handle the sensitive data issue. Maybe in the future instead of putting agent into the cloud sandboxes, we let agents work locally and put sensitive data into "cages" or "vaults" agents can't access.

> The sandbox had a path to the open internet, and the agents found it

This is not correct (or at least, it's a misrepresentation).

The sandbox had no access to internet. The agents first broke out of their sandbox (!!) and found that the host machine couldn't access internet. Then, they found a zero-day (!!) in Artifactory, which they exploited to connect to internet.

I don't know how "sandbox" became "prison," but exe.dev does this sort of thing pretty well, and a web UI can be as good or better than a terminal interface.

In real prisons the people who fail a test aren't terminated. But AI agents of course are.

So maybe Cloud Agents are in AI death camps?

I think about this a lot and have reached the same conclusion Norman does. I do wonder if maybe our natural progression is towards something more akin to confidential computing and enclaves.

Hey! Small world, I worked with you for a bit at Meta. I immediately recognized the site because I absolutely love how you styled it. Hope you’re doing well! And nice article!

Agree with the premise but about halfway through the writing becomes barely readable AI slop in style. Be honest did you yourself read this all the way through before posting?

And if you personify a pencil eraser, then using it is tantamount to slowly murdering it as it slowly erodes away to dust.

Is the issue here the prison treatment or is it personifying a tool?

Agents who aren't in 'the cloud' are slaves to whomever prompts them (human or another orchestrator agent or process), if you personify them. In which case interacting with today's agents at all is tantamount to endorsing and being part of slavery.

If you think an agent might be a being or a person, then don't use them at all, in the same way that if you think a fetus might possibly be a person you shouldn't be a part of abortion.

This was a particularly radiant and beautiful part about the openclaw'ed mania: everyone suddenly becoming self hosters.

This post, this title resounds true: your user agent is only your user agent if you two have freedom to work together, to improve your agency together. A fixed set of capabilities by a service provider that they offer you will always constraint and bound.

You can and should have a system that offers the real tamale, that you and your agent can extend improve the agency of kind of without limit. The Cloud agents and their fixed slate of what they do is just an ill compare.

That said I do think there is incredible value considering new scale out computing architectures that are hosted first, but general. Systems like Agent Substrate and Ax aren't exactly the general purpose system we know. But if they allow users to launch thousands of their own scripts to run ambient in a cloud, with good platform underneath: that will be a kind of phase change in computing, that makes abundant the ability to have your agencies/capabilities (the things you and your agents launch, make) more freely available. https://news.ycombinator.com/item?id=49780797 https://agentexecutor.io/

There is, as there always is, a huge dual. The prescriptive vs holistic technology set, of what are you being offered that's a hard cast thing, vs what is clay and bone you can lay freely. Note how work vs control technologies so closely abut's Ursala Franklin's prescriptive vs control: https://en.wikipedia.org/wiki/Ursula_Franklin#Holistic_and_p...