The Lisp curse, now coming to your ecosystem (javahippie.net)

I wrote this same concern in February: what happens when it's easier to build custom software than spend one search looking for mature libraries?

https://gavinray97.github.io/blog/llm-build-cheaper-than-sea...

Seems a bit hyperbolic to call it a curse. Things like these are on a spectrum for all languages and languages like C and many others are afflicted by ecosystems that are more or less community driven and leverage libraries to a greater or lesser extent. Doesn't really have to do with Lisps per se.

I do think the current ecosystem of open-source libraries won't survive as it is, as AI screws up the incentives too much. But agents recreating the wheel for every project isn't practical either.

What I think we'll see happen is that AI vendors will create private ecosystems for sharing code. I.e. you subscribe to Codex and you get access to a suite of libraries that speed up your development process. Behind the scenes there'll be a mirror of the how the current ecosystem works, with agents submitting bugs and feature requests and a private swarm of agents working the most popular / high priority.

LLMs stop me using open source libraries for two reasons

(1) GitHub is now full of slop. The majority of new open source libraries are now generated by AI. Many come from students who are just looking for resume garnish. Others come from devs who are more excited about being an "open source developer" than actually doing open source development

(2) With AI I can probably build something better. And if it's crap? No problem! I can just wait 6 months and rewrite it with gpt_galaxy or Claude Torah or something

At the same time I have no real incentives to contribute back. Anything I create competes immediately with a sea of slop. Any actual work I do gets identified and swallowed up by the plagiarism machine. And to be honest I only occasionally had good experiences with OSS anyway

I think we are looking at the end of open source software. Not the end of free software - in fact that may end up flourishing with LLMs and reverse engineering and the like. But the idea of programmers sharing code, for communal interest and career building, is completely dead.

So every Lisp developer creates their own time libraries and HTML DOM libraries because it's so easy? No. This is a myth.

It seems like there are advantages as well as problems to this situation. Notably it prevents "monoculture" in software which prevent global worms. If every website implements network protocols from scratch, it would have bugs but possibly not the same bugs and vulnerabilities as it's neighbor. The extreme is every app is its own OS - back to CP/M where running a program involved inserting a disk and turning on the machine, that "interface" had it's charm.

This is rather interesting. But it misses the point somewhat.

The problem is whether or not we want to view code as a medium of communication between other humans vs code as an abstract machine that solves a problem. We know that "programs" in the general sense are fairly objective entities that can be studied objectively, which is where the famous Sussman line about this comes from.

But as a language for expressing execution semantics that a machine can then execute to produce a desirable effect on the real world (a solution to a problem), we would not expect such code to be written with other humans in mind, and as lisp is a strictly more expressive language (both for expressing the meaning of an algorithm that other humans will be able to study and understand and that a machine can execute), and that more people who program are concerned with producing a solution to a problem, the "curse of lisp" is the logical conclusion.

But it's overblown. There exist many excellent and fairly complete libraries for Lisp: Closer MOP, UIOP, ASDF, Bordeaux Threads, etc. And no surprise, these libraries are virtually ubiquitous and considered (at least by me) to be more or less extensions to the language. And these libraries are written in a fairly readable style that anyone can go and try to understand the code.

Secondly, the curse isn't really much of a curse at all. It is more likely to yield more robust programs. The assumption that code reuse through libraries increases software quality doesn't really hold up in practice: libraries that are "complete" tend to add bloat to applications; managing different versions of libraries on the same platform leads to arguments about static vs. dynamic linking, and so on and so forth. Ever had to debug .so versioning bugs on Linux? It's not fun. And yet, that is par for the course if you try to persist in the notion that "libraries and code reuse is always better than reinventing the wheel".

As a reaction to the utter failures that have occurred due to following that ideology, entire programming communities have popped up to fight it, particularly the "Handmade Hero" that became Better Software Conferences as well as "Software Should Work". And while those people aren't a huge fan of Lisp, they really do embody the Lisp mindset of "don't be afraid of building your own system in your own solution's language". And this does usually yield nicer systems to work with because programming is theory building: bringing in different ready-made libraries introduces conflicting theories that requires even more glue code to rectify and the glue code rarely foresees all the discrepancies. And so bugs appear in the integrations between them.

That's not to say that increasing "the curse of lisp" always yields higher quality. The kinds of developers that make great software with that mindset are the ones that are allergic to inconsistency at any level and so squash bugs as quickly as possible and that is what leads their software to be better. But on the practical side of programming work, you feel a lot less inhibited in trying to fix bugs when you're certain you understand the code and how the implementor decided to map the problem domain onto the range of possible solutions. This is where the issue with LLMs comes in: if programming is theory building (which I firmly believe), then what we're really after with using LLMs to generate code is a more effective technique of translating high level theory of the problem into the low level code that executes to solve the problem. As such, maybe vibe coding can work if the vibe coder diligently keeps in their mind what they're after. But usually what that means is writing notes about the desirable properties of the resultant application. And then they're just back to theory building... just at a somewhat higher level.

This is called fragmentation. The advent of slop has caused many repos to stop accepting contributions, and people maintaining their own forks. I do it too, diligently maintaining repos because the original maintainers were either too slow for me, unwilling to add my features, or had moved on.

Since code is now cheap I think it is not the end of the world, but it does suck a bit for users, who may want features from multiple forks.