How OpenAI Used Its Own LLMs to Design Its Jalapeño Chip (spectrum.ieee.org)

Having worked with people doing bringup of specialized chips, I am awed at how the world has changed.

> When the first chips came back from the foundry in May, the team pointed its internal AI models at designing software to run benchmarks such as SemiAnalysis’s InferenceX. On DeepSeek’s multi-head latent attention kernel benchmark, performance climbed from 0.31 percent of the theoretical ceiling (set by the chip’s compute and memory bandwidth) to 88.94 percent in roughly 40 hours. Ho says this result is repeatable, so the time between when foundries deliver the first chips and when production ramps up can be reduced. “All our schedule assumptions are going to be based on the fact we have this capability now,” he says.

With a few handy tips and tricks from apple insiders. But sure, I guess the LLMs helped too.

Seems pretty obvious now that OpenAI is just hyping their models in order to get companies (in this case, chip developers) to use their products in order to learn from their (exfiltrated) IP. Any corporation would be foolish to use any of their or Microsoft’s products, particularly those with valuable IP. There’s nothing in the article that says AI did anything creative but rather that it was used for software development within the overall project. Clear misleading title. Suggest to mark this as clickbait.

IEEE Spectrum is such a good publication. Early in my career I worked at a place where the magazine would be passed around every month with a coversheet listing all us engineers we had to pass it around and sign we had read it. Been a while since I visited the website but love what they did with it.

I grow Jalapeños. This conflation of AI and actual chili peppers irks me.

Aw, I was expecting more details but this just seems to be a rehash of what they unveiled a month ago.

We were able to invent a chip that already existed so fast, you guys.

AI does not make anything new, it is not surprising that it can regurgitate what already exists much faster than humans can invent new things.

At some point people will use an LLM to design an Apple M series competitor.

It is surprising to me that recursive self-improvement seems more plausible now than it did in 2023. Am I the only one to be surprised?

I remember the paper proving that hallucinations could never be fully solved back in 2024: https://arxiv.org/abs/2409.05746

I also remember the hang-wringing about running out of new datasets to train on. Now it appears humans are always generating more data. It's just not as cheap to acquire as legacy data? Meta has to give a deep discount on their API prices to entice people.

I thought back then that humans had a few more breakthroughs in them as meaningful as the seminal Attention is all you need paper. Enough to 100x the capabilities of LLMs back then (10x the smarts and 10x the speed simultaneously).

RSI with a 20 month turnaround for a chip to be made is not exactly breakneck speed though. Physical manufacturing and logistical constraints are going to be and remain a hard obstacle to that process for the foreseeable future.

> Jalapeño can reduce end-to-end latency (the time between prompt to last token) by up to 3.6 times

I'm never sure what on earth this kind of impressionistic math is supposed to tell me. Is the comparison between 4.6 and 1.0? 3.6 and 1.0? Clearly the comparison isn't supposed to be 1.0 and -2.6, even though that's what the words literally mean. I can't be the only person who finds this infuriating and distracting. These numbers shouldn't be impressionistic. They should be precise. That this is an article on spectrum.ieee.org makes the imprecision all the stranger. I'd expect their readershipt to care, for instance, about what's even being measured. Is this the geometric mean of something? The arithmetic mean? And what latency has improved?

So when can we start getting cheap chips? RAM anyone please!

Whatever happened with the Apple lawsuit?

Congrats to the former TPU team

openai should figure out how to make lithography machine, so ASML don't have monopoly on it.