How to Write with an LLM (sockpuppet.org)

Also:

1. It's often a premature extrapolation from diffs and tickets which you could probably generate later, at need, and likely with even better results.

2. If those commit messages can ever influence future work, then it's just carcinogenic cargo-cult cruft. Pruning outputs (future inputs) is important to prevent weird unwanted drift.

But AI commit messages are still bad. Way too verbose and focused on the wrong level: that of code mechanics. That's just wrong. Messages should focus on the level of intent and design, with the primary purpose being to aid human review. They should include a high level overview of the change, decisions, caveats, information not obvious from reading the diff.

So I wrote a skill that captures these principles and allows the agent to even research past related commits and to ask focused questions in order to uncover the intent rather than guessing and writing a bad one.

Now the AI writes better commit messages than it used to, and even writes better than most humans (who can't be bothered to write a good one). Not better than a good manual message, but you can't have everything.

But sometimes even the good writers are tired or didn't think things through. Being able to compare with the AI's version is still useful.

Here is the skill for anyone interested: https://github.com/FooBarWidget/ai-skills-and-principles/blo...

Depends on my "documentation principles" skill: https://github.com/FooBarWidget/ai-skills-and-principles/blo...

You are not a serious person if you recommend that.

Check back in a couple years and I’m sure they’ll be there but today’s frontier models 100% absolutely are not ready to fully own a nontrivial production codebase with no human involvement.

If your work has little liability then you can afford to not care beyond “does it work”.

If you have to worry about quality and ensuring you don’t get sued, you make sure the process works.

If it has to maintainable, you you need the mental model to be present in someone’s head.

Where does this feeling come from? Is it just because the work is complex? We’re all (excepting maybe the amish) comfortable with using at least some automation, be it mechanical or digital, elsewhere in our lives

What’s preventing the editor from using one of your prompts/tooling and replacing you entirely?

There’s zero value in sharing ai-generated output unless your ai has access to something nobody else’s does on which to base things. If all it’s doing is web research you should share your prompt instead. That way people don’t have to deal with ai-written walls of text.

E.g. in one country, 21% of high school students do not even achieve “sufficient reading literacy”…

So the question should be - what are we going to do with this? LLM can produce texts but we as people will not be able to understand anyway…

About PISA research for example here: https://www.edweek.org/leadership/reading-ability-craters-on...

Not that I disagree much, but for myself I have an actual cause connecting the two parts, and it's not something inherent to AI.

Basically, synthetic content is like those fancy desserts - they look nice and the top layer possibly has an exquisite flavor (probably on purpose), but once you cut deeper, it's some weird fruit or nut or other foodstuff that belongs next to potatoes and meat, not in a cake. So I learned to avoid such desserts and stick to basic stuff I know are good entirely, like IDK a cream pie or cream-filled chocolate stuff or such.

Same is with content. My problem isn't that it's AI, but that it's wrong, full of mistakes human authors don't make. Hallucinations mid-text that make half of it convincingly argue bullshit. Or the uncanny or repetitive factor in art that kicks in after few seconds of looking.

Like with desert, I'm avoiding disappointment and wasted hopes. When synthetic context gets better, I'll be consuming it just like "vanilla" one, for the same reasons.

I love reading, but I mostly read non-fiction and I'm a bit of an amateur history buff. Last time I read a fiction book I was left feeling like I was preached to by an alien about how life doesn't really matter (sci-fi authors will often do that). The state of literature is pretty dire.

The problem here is that the volume of low-mid content is making it harder to find the best content.

One person was arguing that they need llm because english isn't their first language and then it helps them write better and be understood. I was incapable of reading their (extremely long) email of arguments; I tried but my mind just went elsewhere as my eyes were moving through the lines.

At the very end there was a disclaimer that llm had been used to "aid" in writing that very email.

Poor guy thinks he's communicating better but all he's doing is making his emails completely unreadable. It's a sad state of affairs.

It often makes them less informative and a lot less sincere.

I know lots of AI-enabled writers believe this, but I've never actually seen it be true.

I mean, how can you say that he didn't think it himself? I thought this same thing and realised apparently I'm not the first. You have no idea how many of my blog posts ideas have come in HN first before I could publish. People can have the same thoughts. And have em all the time as well.

Psss.. See how @sm-silversight who commented on your post said the same thing I did? That's realtime example right there.

Coming back to the topic at hand. I agree with the OP. If you don't care to write it, why should anyone read it? AI has good usecases. But am not sure writing is one.

Writing is not just about the end result. It's also about the thought process. I learn about the topic more as I write. I correct my incorrect understandings as I write. When am pissed, my article has a different tone. Am rude, too direct. When am content/happy, different one. All of this is sterilised into something robotic where the author won't even remember what he/she wrote? What's the to like? For brownie points?

That's more or less the same thing expressed more nihilistically. But nihilism is, like, just a word man (to 'The Dude' it up a bit). Perhaps we are either fiddling on the deck of the Titanic, or just staring at the waves?

What do you think the results will be of the machines consuming only their own slop and Habsburging themselves intellectually, and then all the humans abandoning any attempt to either talk or listen? As a global civilization of humans? I'm not sure we've explored the likely consequences of 'if you can't spend them time to write it why should anyone read it'.

Maybe it's gone too far already.

Couple weeks ago I had someone linking to a doc page in my ticket "Made this summary, hope it helps". It was 100% AI generated, the person clearly had no understanding of the problem, it only mentioned UI changes although the thing required changes in many systems. Complete waste of time, I don't think I will humor anyone who makes AI summaries anymore.

Is there anybody on this site who isn't?

For instance, if I said the grass is green vs if I said the grass looks nice vs if I said the grass is doing well and looks how it should vs if I said in my opinion the grass is healthier than its ever been.

People are beginning to wisen to how to more skillfully deploy generative models.

When you review code, you're not taking it all in end to end like an email or memo or blog, you're looking for important symbols, recognizing structures and patterns, jumping between different files at different levels of reference and abstraction, and building a model in your head of the principles that make it work and their consequences for the overall procedure.

Code is "meant to be read" in the same way as a law, blueprint, policy document, or manual is, ie: jumping between relevant sections to get small pieces of information relevant to some task. The act of "reading" never starts at the beginning, it starts with a ctrl-f or flipping through an index or table of contents for a specific relevant symbol that is the only reason you are looking at the document.

Code is invention, not art (and should be covered by patent law, not copyright law, I will die on this hill)

It’s perfectly possible to consider e.g. working, LLM generated, human approved code acceptable, while considering LLM writing that hasn’t undergone major human editing unacceptable because of its repetitious, formulaic, marketing-oriented style - a criticism that doesn’t even make sense for LLM generated code.

A lot of documentation generated in corporations has marginal real value.

> If your commands are bulleted instead of numbered and code blocked, it's wrong.

That’s easy to instruct an agent to do. Put it in a skill.

> If you didn't crawl through the playbook, it's immoral to hand that to me, you're wasting my time with untested slop.

Using an AI doesn’t absolve the user of their responsibilities. This is an easy problem to solve, just make sure teams know what’s expected of them and encourage everyone to push back (professionally) against offenders.

it is great at analyzing the argument, finding inconsistencies, helping you think through what parts should be cut, helping you refine examples or fix the occasional “how do i get this phrase to work correctly in this transition?” kinds of stuff. but anything where LLMs are the prima materia… that stuff literally only makes sense _to me_. which makes sense, because it is written _for_ me, no matter what instructions i actually give it, bc of memories and a million other things. and i say this as a complete maximalist wrt. trying to use llms for absolutely every last thing they possibly can be used for, just to see what it’s like.

i guess i would say that it does very, very little to make the writing process meaningfully faster; it _can_ do _plenty_ to help make your output better though, which is definitely something— just isn’t the thing most people are looking for.

Style guides are a big tools people are missing out on. It isn't enough to say "write concisely and technically". Give them a sample. Yea verily these many 5 or 6 years ago, "style transfer" was a big thing that early LLM tech was doing. It's still very good at it.

That said I still tend to cut out at least 25% of the resulting verbiage and adding back another 10% or so more of my own original content even under those circumstances.

Where it has been really helpful is that I tend to want to write in a conversational style that doesn't seem to match most people's expectations of a technical document. The LLMs let me write my way and style-shift it into something closer to what people expect. And LLMs, since let's be honest they're the primary audience nowadays. Which I am not even upset about; I'd rather 5 LLMs read the architecture document than the amortized .2 or so humans I could expect in the same circumstance 5 years ago.

Which also implies, in many cases, I am feeding the AI as much text as I expect to come out, or in some cases, even more, as I am describing context, reasoning, and other things that may impact the writing but are not necessarily repeated in the final text. I factored out a lot of the context into my user-level CLAUDE.md which has helped cut that down a bit.

New areas are where it is most dangerous to use an LLM, as a person can’t tell when it doesn’t align with their views (because they don’t have any yet), is misleading, or flat out wrong.

If I want to know what an LLM thinks, I’ll ask an LLM. If I’m reading a paper from a human, it’s, because I want to know what that human thinks. If they need the LLM to fill in the gaps in their knowledge, then they don’t know enough to write about the topic yet. Doing that work to write something themselves could help move them learn enough to write something that is hopefully worth reading.

Explaining difficult topics or summarizing to you is fine. But that's a world of difference from having an LLM write for you. You THINK the LLM is helping you write above your level, but that's because you are not in a position to tell it's bad.

$ chatgpt make this comment more glib

Charitably, they can be thinking that it's fine to have LLMs organize information intended for LLMs.

I can see being lazy and skipping it, but refusing to use them even for more minimal advice purposes seems crazy to me.

Clearly I am not a gifted writer as my original comment failed to convey this very well. And now I am pondering, maybe next time I should put my comment in an LLM to review it first. This meta lesson is playing out over and over everywhere by anyone writing anything.

I am in no way saying LLM writing in general is good, but ignoring the reality of what is happening is silly. For good or bad the not great writers not only will, but are doing things like voice rambling into an LLM as a starting point and editing it from there.