The foolishness of "natural language programming" (1978) (cs.utexas.edu)

143 pointsby xiaoyu20068 hours ago85 comments

nickm12 7 hours ago

I send this article to everyone who tells me how English is now a high-level programming language, the next level of abstraction up the chain from machine language to <their favorite language>. This analogy tends to be followed by "Just like most programmers never look at assembly code, looking at higher-level source code is going to become something only a few people do".

Dijkstra's arguments tend not to change their minds. I think the analogy takes hold because there is a kernel of truth in it. AI models now give us the magical ability to program in natural language, but the implication that an AI model is a form of compiler is deeply flawed. AI models don't replace compilers, they replace software developers. The reason software developers rarely look at assembly code is that compilers (and emulators and VMs) are able to precisely preserve the semantics of the original program.

A natural language spec is not a program, and there are many possible programs that can meet the spec. Software engineers and AI models turn specs into code, but the code is not the spec I think the analogy clouds our thinking about the value of programming languages as a tool for formally describing what the computer should do and enforcing properties of the program that should not be left to either humans or AI models to uphold.

indrex 6 hours ago

You mean the English to code layer is stochastic while code to assembly is not?

This stochastic nature existed before AI. Humans are stochastic too. Two engineers convert specs to code in two different ways. I would argue the distribution is wider among humans than among agents.

DonaldPShimoda 6 hours ago

Arguing that two humans' distinct interpretations of a single specification is stochastic is patently absurd and requires, I think, some shred of evidence.

I would argue that, in the absence of such evidence, we cannot conclude that the English-to-code conversion process is stochastic; it is merely not deterministic, which is altogether different.

CamperBob2 5 hours ago

Would they write the same code, if given the same spec? No? How about over multiple trials? No? Even the individuals' own code would differ from one trial to the next?

Well, then, their interpretations of the spec are stochastic. Good enough for me, anyway.

DonaldPShimoda 2 hours ago

That's not what "stochastic" means. You might want to look that up before you keep (mis)using it in conversations about actual stochastic processes.

nickm12 6 hours ago

As I said, "AI models don't replace compilers, they replace software developers" so yes, I agree that AI models are like humans.

while both LLMs and humans are not 100% deterministic, I don't think that's the interesting part. It's both true that there are many assembly language programs that implement a C program and that there are many valid programs (in any programming language) that correctly implement a natural language spec. But there's a significant qualitative difference between the two types of translation and that would be true regardless of whether the translations are stochastic.

sorokod 32 minutes ago

The stement to which we are replying will (has?) launched a 1000 startups.

It would be interesting to state sane measurments that determine a "distance" between a spec and software that claims to implement it

sorokod 4 hours ago

I would argue the distribution is wider among humans than among agents.

Please go ahead and do.

chrisjj an hour ago

> Two engineers convert specs to code in two different ways

So? Two compilers convert source to binary in two different ways.

glimshe 5 hours ago

Your acquaintances aren't changing their minds because the arguments are weak. Both the article and your comment assume that the only mode of interaction for natural language is a very high level description of programmer intent.

You can interact with AI and say "code me a calculator" or "now declare a array of booleans to map to each of the keys in the UI and set the corresponding array position to true when the key is pressed".

These commands are written in natural language at such different levels of abstraction that is hard to put them in the same bucket. It would be fair to call the latter an imperative language with the LLM in the role of a compiler. That's very similar to how I use AI.

soco 18 minutes ago

The latter is also very far from specifying a calculator. And based on what we see in the "Show HN" projects, also very far from what the average person process is of using AI for generating software.

blurbleblurble 5 hours ago

The LLMs don't replace compilers, they challenge the next generation of compilers. And they don't replace software engineers, they shift them into yet another metasystem.

jason1cho an hour ago

Why do compilers worry about the challenge? Just call an LLM internally from a compiler if LLMs are better than compilers.

In my opinion, LLMs don't challenge anything.

bulbar 2 hours ago

> AI models don't replace compilers, they replace software developers.

No, they replace the need for a precise description of what you want the computer to do by a mechanism that's really good to fill in the gaps. But guesswork only takes you so far.

pastel8739 2 hours ago

Is that not what software engineers do?

mettamage 24 minutes ago

I recently vibe coded an application fully and tested it with users. So all the functional requirements were there.

Guess what wasn’t?

The nonfunctional requirements.

Also: my app needs to do LLM classification. But instead sometimes the coding assistant wrote not an LLM api call but it wrote a regex to classify instead. This was despite the fact that I mentioned to not use regexes to classify things with.

Non-technical people don’t know what a regex is. Usually they also aren’t knowledgeable enough about nonfunctional requirements.

All of this may improve over time of course. But right now, I am not impressed (in terms of replacing SWEs).

sadilek 2 hours ago

Agree. For me, that's exactly what happened. I work with the agents the way I'd work with a team of developers, not the way I use a compiler. Since the end of September, they've made about 970 commits to a project of mine, and I haven't read a line of its code.

And like with a team, many programs could meet what I asked for, but I check the behavior of what they built, not the code directly. I think of it as working on the level of an engineering director, who also doesn't review code directly. Each agent shows me its change running in the app, in a short narrated screen recording, like in a launch review for the director. When it's not what I meant, I say what, and it goes another round.

jason1cho an hour ago

If you like being a software tester or director, go ahead with it.

However you can't prevent some other people from working as software developers who write and read code.

Even with cars, you still walk. Even with factories, someone still craft handmade bags.

sadilek 24 minutes ago

I can't prevent them, and in fact, I don't want to. Didn't mean to imply that.

chamomeal 38 minutes ago

Thank you for articulating this!! This idea has been rolling around for a while but I haven't been able to put it into words.

I think "I'm working in a higher level of abstraction" is easier for some devs to say than "I asked somebody else to do the thinking and the work." I don't think there's anything wrong with asking an LLM to do the work. But pasting jira tickets into claude is not abstraction lmao.

I guess arguments on either side of this can get difficult for a reason that another commenter already mentioned: you can use LLMs on a wide spectrum of granularity. You can one-shot a whole project, or you can tell it to rename some vars in a single file. I guess there are uses that are probably closer to programming. But if you're not even looking at the code, I'd say that's closer to somebody else doing the work

eru 25 minutes ago

> The reason software developers rarely look at assembly code is that compilers (and emulators and VMs) are able to precisely preserve the semantics of the original program.

'Precisely' is doing a lot of work here. Depending on your language some things might be preserved and some not. Eg if you write stuff in Haskell, you don't really preserve much of anything about memory usage, you leave that to the runtime system to figure out, and exactly how the GC works isn't really specified in the language standard.

froh 2 hours ago

I think the difference is between communicating to a purely mechanical being versus communicating to an intelligent entity: unlike programs, which are purely formal languages, we write textbooks and papers in a mix of human language and formal language. often we introduce "just formal enough" notations. and the intelligent reader will read in-between, inter legere literally means just that: reading in between. filling the gaps, connecting the dots.

and in practice I find it _very_ effective to express key ideas with some formal language where applicable. a little ebnf here, some snippets of python or prolog there or other languages there. and I find llms wonderful at filling in the dots and, and that's the topic here, to translate our semi formal conversation into rigorous, formal language.

and I dare one step further: I also find llms useful in formulating and refining technical proposals, for human peer review and llm driven implementation, and of course these proposals mix natural language prose with semi-formal and formal detail. think PEP-12 and PEP-1 style. and these then get work packages which each loop in a spec as in sdd.

seems to work just fine so far :-)

uriegas 5 hours ago

I've had the same view since coding agents appeared. LLMs are good for summarizing, thus I tend to use them to understand code. However, the more I understand a code base the more I tend to be more picky with the agent. While the less I know about the code base, the more open I am to let it do its thing.

I've realized that the agent doesn't write 'good' code, instead my vague understanding of what I expect makes me think it does. When I look at the AI code, I see that there are more nuances and decisions that needed to be taken before writing the code. One would say that a spec would suffice, but then I remember TLA+, developed by Leslie Lamport to help developers think critically about their programs/algorithms. Sitting down and formally writing what I want to do helps me better understand the problem and come up with a real -not vaguely defined- solution.

imhoguy an hour ago

Todays natural language program won't work tomorrow due to unstable nature of natural language.

Apart of natural evolution of meanings, look at Orwellian limitations some (cloud) models impose on is that our natural language program become illegal. Due to topic or combination of words the program needs to be flagged and soon likely reported to three leter agency.

Next, RMS would ask: can you make your own natural language program compiler from scratch? I know we have (very expensive) local models, but wait until GPUs get a propper online subscriptions to work and surveillance taps.

GianFabien 7 hours ago

Natural language is full of nuances and implicit cultural and domain specific connotations. LLMs work because they have been trained on massive corpus of materials that encode some (but not all) of those hidden meanings.

With specification driven development, the prompts need to be very detailed in order to produce exactly what is required.

skeledrew 7 hours ago

Humans also work because they've been trained on a massive corpus of materials, spanning like an average 15 year period or more. And humans too need to be prompted, sometimes everyday, with meetings and whatnot to produce exactly what is required.

LoganDark 7 hours ago

I used to do software dev without meetings. It worked just fine. I would send whatever I did once it was done, and get direction on where to take it next. Maybe this breaks down in companies with more than a few leaders, or where the leaders can't otherwise keep in sync.

Also gave me plenty of opportunities to go above and beyond. I miss when that felt safe.

andsoitis 7 hours ago

> I used to do software dev without meetings. It worked just fine. I would send whatever I did once it was done, and get direction on where to take it next.

You didn’t collaborate with other engineers? Or what if the spec you were given was unclear in places? Forgive me, but the style you describe sounds incredibly collaborative, super transactional.

LoganDark 6 hours ago

Funny story actually, there were a few other engineers but not very much collaboration. They were resistant to sharing source code or algorithm details, so I had to essentially reverse engineer how their other products worked. I was good enough at it that I came up with several features they didn't know were possible. They eventually ended up letting me go and cloned my design, which I thought was weird, but it was developed for them in the first place so not a big deal.

Exoristos 6 hours ago

The lengths to which devs will go not to collaborate.

LoganDark 5 hours ago

To be fair, it's a very secretive industry, and betrayals are quite common. Worse, a lot of those betrayals are done on pure impulse without even thinking through the consequences first, so you can't actually build incentives against it because they'll just be ignored.

skeledrew 6 hours ago

Maybe those were some very detailed requirements you were getting, the deliveries were straight-forward, or you already knew the space pretty well. Or you're the only dev there. Somehow most teams just keep tripping over each other and need near constant management to ensure they're delivering what the stakeholders expect.

LoganDark 6 hours ago

They weren't particularly detailed, but I did know the space quite well. I was a heavy user of other products of that type, and knew very well what I wanted out of one.

dahart 6 hours ago

This depends entirely on how well the requirements have been communicated, right? If you could go for long periods without meetings on large projects, it means either you had great PMs or you’re very good at collecting relevant info and planning in advance. Or, it could mean that your target was quite flexible and open to interpretation. I don’t doubt it, and I’ve known some developers who could do what you did, deliver something great without that much back and forth. I’ve also known many developers (probably most TBH) who will focus on problems the customers don’t have and endlessly go down rabbit holes that have no bottom. Heads down software engineering for big-O and data structures seems to be a lot more fun for some people than fixing weird hard to repro bugs with vague repro instructions that customers complain about. Many very good devs still need to check in and iterate and get some steering, especially if the requirements have not been well or fully explained… which in my own experience has been more or less always. ;)

LoganDark 6 hours ago

Maybe my definition of meetings is different than yours. To me, meetings are voice or video calls where multiple people talk about progress and direction, raise issues to the others, etc. and the leaders can make calls across the whole team at once. Staying in constant communication doesn't count as much because it's more async and incremental.

skydhash 6 hours ago

> Humans also work because they've been trained on a massive corpus of materials,

Massive? My introduction to programming (from noob to beginner level) has been like only 10 books worth of content. Including some alternative pathways like 3D, security, system administration,... To this day, I don't think I've reached 50 books of materials. And If I counted everything it would be still small.

Human learning is very efficient (not time wise, but cost wise, and even training data wise).

senordevnyc 6 hours ago

Only if you ignore the billion years of pretraining…

skeledrew 6 hours ago

You're forgetting all the schooling before that, which taught you how to read, do math, etc. And the general daily experiences of just being a human being interacting with the world. Magnitudes more data taken in than any language model.

DonaldPShimoda 6 hours ago

> Magnitudes more data taken in than any language model.

Ah, yes, I can fondly recall reading and internalizing every word available on the internet when I was young...

nvme0n1p1 6 hours ago

A picture is worth a thousand words. How much video data was your brain trained on?

skeledrew 6 hours ago

You should check out how much raw data just your eyes process for every second they're open (although a significant amount is discarded before it gets to the vision center, but still). Then consider the other senses also constantly doing their thing.

LoganDark 6 hours ago

The vast majority of people don't internalize most things. Some internalize even less. It can become a big issue with things like dissociative disorders. (I have some experience with that)

groundzeros2015 7 hours ago

Yep, LLMs work to the degree you don’t care shout the choices made. That’s why it’s great for demos and starter templates.

But for mature software eventually every single detail matters.

montag 7 hours ago

After reading this essay, I wonder: what if program code was withheld from training data? Then how useful would LLMs be for programming?

bob1029 2 hours ago

NL programming is foolish to me not on technical grounds but on practical ones. It is most valuable when the business wants to understand how an already working thing does its job.

I have observed several samples where the business, when provided with a natural language configuration opportunity, will wind up staring blankly at the textarea until the original developer gives them a complete starting point.

I don't think "natural" language makes much of a difference unless you are already skilled in the art of developing useful engineering specifications. I often have to babysit the business through very basic boolean algebra concerns despite everyone having 3+ decades experience dealing with this stuff. Making everyone speak python could only make this worse, not better.

I use HN to keep myself warmed up on natural language expression. The difference between developers who post here prolifically and those who barely write one email a month is fairly shocking in my experience. If you intend to use a specific instrument to accomplish a complex task, you need to practice it constantly. Ideally every single day. There is no substitute for this. There never will be.

mpweiher an hour ago

Possibly relevant:

The experiment in designing a language that resembled natural languages (English and Japanese) was not successful. -[1]- William R. Cook, AppleScript, HOPL IV

Now our new LLMs are different, because they don't use "language the resembled natural languages", but actual natural language. But the lesson is instructive nevertheless, because it gives us some empirical evidence to compare to Dijkstra's theoretical arguments.

And this is that natural language is at best a two-edged sword. Yes, there are benefits, but there are also huge drawbacks. And one problem is that the benefits are front-loaded and the drawbacks are long-term problems.

To me, one thing we are missing is that what LLMs do for us actually has 2 parts.

1. It allows us to use natural language, allowing us to be fuzzy.

2. It allows us to specify what we want at a much, much higher level than usual.

With our LLMs, these two are intrinsically linked. I think the whole would be much, much more valuable if we could separate them.

[1] https://dl.acm.org/doi/10.1145/1238844.1238845

riknos314 6 hours ago

> I suspect that machines to be programmed in our native tongues —be it Dutch, English, American, French, German, or Swahili— are as damned difficult to make as they would be to use.

He's still quite correct. I generally see a machine as being "programmed in" whatever is tracked in source control. So as long as your git repo is tracking changes in Java, that's still programmed in Java, even if the Java was generated with the assistance of LLMs. I may consider the thesis of this article invalidated only when the majority of repos start tracking natural language rather than programming languages as the primary artifacts defining behavior of the software.

kikdij 39 minutes ago

His point is moot. His argument considers that natural language would be the direct expression of the program. It's not. It's an abstraction layer, that is then converted in formal language. He could have foreseen that, given he takes explicitly the example of machine code and high-level languages. High-level languages do not replace machine code, they translate into it. Just like natural language programming translates into high-level language.

hintymad 6 hours ago

What Dijkstra said still holds true, and the essence is consistent with Brooks' discussion on essential complexity vs accidental complexity. I do think one thing has changed, though: when someone asks AI to implement XXX with very vague English, the implicit ask is picking the best[1] ever solution that someone else has done to solve XXX. For instance, if I ask AI to extract the Japanese subtitle of Shirojima show and convert it to an epub with all the nantural and cultural explanations, I was assuming that AI will fill in the blanks and do a good job and AI did. This won't help expert, but does it matter? Being good enough can be good enough to take away many people's jobs.

[1] For many people, the result is actually the mean not the best, but it's another topic.

teleforce 3 hours ago

>We would need all the intellect in the world to get the interface narrow enough to be usable,

It seems Dijkstra is describing potential calability of LLM model trained on published collective human knowledge enabled by the Internet

>and, in view of the history of mankind, it may not be overly pessimistic to guess that to do the job well enough would require again a few thousand years

Since the article was written in 1978 (about 50 years ago) and from the time the computing resources of CPU and RAM has increased exponentially until the LLM first appeared in 2020s, it seems the thousand of years prediction is merely several decades accelerated by the rapid advances of computer and communication systems technology spearheaded by companies like Nvidia.

wseqyrku 5 hours ago

LLM interfaces on everything these days equate to touch screen on cars. It is definitely a useful tool, but generally speaking, that's not how you'd talk to your computer.

hbarka 7 hours ago

There’s em dashes in that 1978 article.

timcobb 7 hours ago

It can only mean one thing.

gnabgib 7 hours ago

Get out the Binford 6000, it's Tool Time?

soulofmischief 7 hours ago

Unreadable.

kevin_thibedeau 7 hours ago

The typed original [1] has double hyphens that have been converted to dashes. His typewriter (Hermes Ambassador) was fixed pitch.

[1] https://www.cs.utexas.edu/~EWD/ewd06xx/EWD667.PDF

voxelghost 6 hours ago

Wasnt the original famously in written with Fountain pen? And the typed out version reproduced much later?

kevin_thibedeau 6 hours ago

He typed them until 1979 when his typewriter stopped working correctly.

voxelghost 4 hours ago

Well - in this case the EWD in TFA - has a footer that says.

    transcribed by Tristram Brelstaff
    revised Fri, 19 Nov 2010

And so seem to be the case with most of the other EWDs I looked at as well.

jtbaker 6 hours ago

His namesake algo was also written entirely in his head in about 20 minutes at a coffee shop! https://harlepengren.com/dijkstras-algorithm-20-minutes/

galkk 3 hours ago

I'm not native speaker but the TOC smells llm to me

DonHopkins 5 hours ago

Luca Cardelli created the handwritten typeface Dijkstra font in the late 1980s, to mimic the distinctive and precise handwriting of his colleague Edsger W. Dijkstra.

It shipped as a free PostScript font with NeWS:

https://www.donhopkins.com/home/NeWS/fonts/Dijkstra.ps

https://lucacardelli.name/indexartifacts.html

(You have to click on the "Fonts" tab, there's no direct link to it.)

Dijkstra [.ttf (2013 updated version by Reuben Thomas)] [.zip (old PC)] [.hqx (old Mac)] [Preview.pdf]

A beautiful handwriting font of unknown origin, first popularized by E.W.Dijkstra. Excellent for slides that appear to be carefully written, and for short, deep technical notes.

https://lucacardelli.name/Artifacts/Fonts/Pc/dijkstra.ttf

https://lucacardelli.name/Artifacts/Fonts/Pc/Dijkstra.zip

https://lucacardelli.name/Artifacts/Fonts/Mac/Dijkstra.sea.h...

https://lucacardelli.name/Artifacts/Fonts/Dijkstra.pdf

retrac 6 hours ago

Oh look at the last paragraph on the first page. Manually justified:

           In order to make machines significantly easier to use, it has been
    proposed (to try) to design machines that we could instruct in our native
    tongues.  This would, admittedly, make the machines much more complicated,
    but, it was argued, by letting the machine carry a larger share of the
    burden, life would become easier for us.  It sounds sensible provided you
    blame the obligation to use a formal symbolism as the source of your dif-
    ficulties.

How does one even achieve that with a manual typewriter?

kevin_thibedeau 4 hours ago

It's coincidence in this case. Proportional typewriters were made but this wasn't one of them.

archargelod 6 hours ago

It's not just em dashes, it's 1: typewriter em dashes aka `--` and 2: usage of them in pairs, like parenthesis, that is completely different from how LLMs use them.

adamrezich 6 hours ago

Literally stop the presses

nickm12 6 hours ago

Proof that Djikstra was an AI model!

ThrowawayR2 6 hours ago

Dijkstra elegantly explains why AI systems are effective at mathematics and programming but much less effective when writing prose or other realms where what is good is nebulously defined:

> "The virtue of formal texts is that their manipulations, in order to be legitimate, need to satisfy only a few simple rules; they are, when you come to think of it, an amazingly effective tool for ruling out all sorts of nonsense that, when we use our native tongues, are almost impossible to avoid."

computerfriend 2 hours ago

I am surprised that cs.utexas.edu uses a TLS certificate chain who's root is eMudhra's "emSign Root TLS CA - G1".

sam_lowry_ an hour ago

US institutions are on eMudhra certificates through inCommon, yeah.

AFAIU this particular chain is not compromised.

jeanpah 7 hours ago

This article shows every now and then, and it's a must read. Indeed the natural language programming requires massive clusters of gpus just to make a half baked crud app

likoya 6 hours ago

I find c++ very unnatural, since it's a giant collection of small rules in the end more complex than assembler. I can't remember the entire sintaxis, zoo of function decorators, undefined behavior, intrinsics which are just labels over asm instructions, Code golf sintaxis...

chrisjj 42 minutes ago

A student one told me he wad struggling to get his head around C++. I advised him to stop, since success would leave his head in an unusable shape.

darrinm 2 hours ago

App stores are approving an order of magnitude increase of new apps because people are now able to create them using natural language. End users are deriving great value from them. So this is settled?

batshit_beaver 2 hours ago

The first sentence is easily verifiable, but the second carries the burden of proof.

GarnetFloride 5 hours ago

Fun fact: there are over 600 definitions of the word "run" and over 400 for "set". But yeah, English is really precise enough for programming.

How many times have we seen that there's been a spec for something and the programmers write something that is exactly like the spec said but was completely wrong?

Brian_K_White 5 hours ago

Natural language as code has existed for thousands of years. Today we call it legalese.

It's excruciating to write, excruciating to read, and even legit geniuses with decades of hard won experience still leave room for essentially infinite interpretation. laws and contracts are all still swiss cheese and can be made to demand or prohibit whatever the party with the most money or other influence wants.

AgentMatt 2 hours ago

If law was code, there would be no need for judges to make decisions. Instead, a "judge" would be someone making a mechanical derivation of the judgement given the situation and the law.

The law is rather a complex set of guidelines which need to leave some space for interpretation, so that additional relevant information for the situation at hand can affect the decision.

anonymous908213 7 hours ago

> (You have only to look at the indeed alarming amount of on close reading meaningless verbiage in scientific articles, technical reports, government publications etc.)

It is perhaps for the best that Djikstra did not live long enough to witness this problem become orders of magnitude worse.

xiaoyu2006 8 hours ago

A really interesting read in the current era.

mjb 6 hours ago

And yet, the specification of almost every program in existence was communicated in natural language. At least between the customer and developer.

This argument makes the implicit assumption that humans are uniquely able to convert between this "customer wants a program that solves their problem" level and compileable code. That was a reasonable thing to believe in 1978, but is not a reasonable thing to believe in 2026.

likoya 3 hours ago

Once an employer asked me to automatize his work, so the task that he did in weeks was done in seconds. I made a gui that showed every step he did manually, so he could check and understand every step. And yet he became confused and lost track of what the program was doing. He frequently asked me the very concepts he taught me so I could do the program. I just implemented exactly his specification. I didn't created any new concept, but as soon as the task became automatized, he stopped understanding.

anthk 6 hours ago

Inform7 disagrees; altough you can use Inform6 which is like a really simple OOP language to create text adventures.

beej71 5 hours ago

I appreciate inform7 is quite human-readable, but it's not free form.

veqq 4 hours ago

Another article on this topic https://alexalejandre.com/programming/no-end-of-programming-...