OpenTPU – An open-source AI accelerator, developed by AI (github.com)

If I now tell a machine "Do what you think is best, and keep doing it forever.", have I now created a machine that can do stuff? If I later die, who will be responsible if the machine changes its strategy?

How far back do you look in the action chain? If an LLM I start today starts an LLM that starts an LLM that starts an LLM that ... 100000 levels deep and 100000 years in the future, is it still my fault? If so, everything I do today is a lungfish's fault, not mine.

Sure, but until we can turn that tautology into something more rigorous, we can't tell if the thing humans do is more or less than what some arbitrary non-human (machine, animal, or eventually perhaps alien) does.

How are you measuring complexity here? How are you measuring inputs? Your average llm is trained on a corpus that vastly exceeds the amount of data I could read in my lifetime.

2024 called, it wants its talking points back. I don't think claims like these are defensible after all the progress we have witnessed in the last year alone.

https://en.wikipedia.org/wiki/Unmoved_mover

One can stand up an agent like openclaw and prompt it with something very general. That would kick off a recurring loop that could indeed see the agent decide for itself it needs to design new chip hardware. Technically a human kick started that loop, but when does that stop being important? I don't attribute all of my actions and decisions to the fact that my parents brought me into this world, for example, and I'd hope they aren't legally on the hook if I screw up.

Not in the same way. The only thing an oven can do by itself is thermostatic.

All machine learning (LLMs included, but way broader than that) is bad at learning compared to any organic brain, to the extent that any organism this bad would starve before learning to eat. However, even a human can't become a chef unless trained, we learn a lot more than we innovate, and what we happen to want without prompting is not generally well aligned with what is desired by people who pay us, which is why we need all those boring workplace things like "a boss".

But even then, this diversion is like saying "an oven can't cook a meal" in response to someone saying they built an oven and "it cooked a meal". Like, it's obvious they didn't mean it did every step by itself without anyone ever even bothering to ask it to: if they meant that version, they'd be a lot louder about it.

> We get around that with training data but there will always be things with no/less data or outdated knowledge.

And? The linked git page (implicitly) claims that there is sufficient training data to do this task.

It may, of course, be wrong. I won't be surprised if it turns out this simulation is too far from reality. But the claim is "it does ${thing} now", not "it's generally intelligent and can do everything now".

huh.

Neither can you. If I drop your ass off in the woods at a few days old, you're back to 10,000 BC, hell more like 200,000 BC. So I don't get why you have these weird pendetic responses that are completely out of scope.

Also, a huge portion of AI training these days has nothing to do with humans, AI trains AI.

>there will always be things with no/less data or outdated knowledge.

And guess what, you're not doing them either! HN posters keep acting like humans are an island, but nothing in the modern world works without a society. Once you put an AI in a harness that can ask questions it isn't really much different than you.

Additionally, it's not clear how well large mask roms scale. For instance the Nintendo switch cartridges were expected to be mask roms, but instead are Macronix's XtraRom technology, which is essentially a flash cell array made denser by removing the erase functionality. So basically the die gets manufactured with all bits at the same state, a late manufacturing step either empties or fills the floating gate of the bits you want different, and then it's treated as pretty close to a mask rom. It's not even clear if the bits can be changed without a bare die, a floating probe array, and specialized hardware. Though, like flash the electrons in the floating gates will eventually tunnel and cause the data to bitrot.

So from that it appears that even at the tens of millions of chips volumes that would make sense for essentially whatever size of maskrom, the memory manufacturers tap out at 128megabit for a mask rom chip, and push you towards something flash esque.

And at the end of the day, flash without the erase functionality is pretty damn close to a mask ROM, and lets you write it near the end of manufacturing rather than at the something close to the metal 1 layer.

You get an FPGA for timing. They're less capable, but (in many common design architectures), they output their results once per clock, every clock, on time, every time. If you can hit a fabric clock of say 100MHz, clocking all the weird logic you can stuff in there, it gives 100 million outputs per second, never skipping a single one for any reason (short of total failure). The penalty is that making a small change to your desired "program" can be very expensive, and many things won't be realistically possible at all. Or at least won't fit into a part that you can buy. But things like audio, video, and high-frequency trading love being able to guarantee timing.

(Of course there are other ways to write your FPGA HDL, but that's one of the more common ones. And you do see DDR-style clocking, and similar, every now and then.)

That would be better suited to FPAAs (field programmable analog arrays). FPGAs can usually only work with clocked digital signals.

What resources are unbounded? There are limits to growth in the real world, how are these ASIs going to escape physical reality?

That sounds like a real hassle. Isn't it best solved by wireheading (subverting one's own sensors or reward system), which is much less of a hassle and can get one's utility function as high as desired?

That could be prevented by engineering hard limits that can't be circumvented by the AI. But that sounds very close to the same "do what I mean" problem as "do this but don't actually kill us or drug us".

"Super intelligence" only means super ability to predict outcomes. It's mathematically equivalent to data compression (gzip is a very primitive AI), and it's entirely orthogonal to ethics.

No, it's just you conflating intelligence with other things. Humans are "super intelligent" compared to every other living creature on the planet, and yet we've driven more other species to extinction than everything else other than the most major extinctions (and we're still going full blast at it).

Morals, ethics, and suffering are relatively measured systems. For example AI or some alien could reasonably think that putting us all out of our misery would be a more moral solution than letting us live. Or that getting rid of humans is more moral than letting us wipe the rest of life off the planet.

This is the key concept of alignment. Ensuring that if you build something more powerful than you, that it aligns with what you want instead of what it thinks would be better for you.