> I'm an engineer at Meta and I never felt this uncomfortable doing my job. I'm writing code these days and it all feels like a black box. This new standard of shipping with AI is moving so fast that if we're going to spend our time checking each line of code, it's going to take forever. That means you can end up with systems you've built where you don't truly understand how all the code works. You understand the high level, but not every detail. And maybe this is becoming the new standard. It goes against every instinct of being a software engineer. I'm used to understanding what I'm building. Now I'm trying to figure out how much I need to understand before I'm comfortable moving forward. Are other engineers feeling this too?
How are you handling this?
bitwizeshift 2 hours ago
I’m surprised I haven’t see anyone say this yet, but I don’t let the LLM design the code at all, I let it implement a design I have created.
By that I mean that, rather than let the LLM do the entire structure, I take the time to create the full hierarchy of abstractions and actors that I want my code to do. I decide, at the UML level, how abstractions will sit and interact — and then I make the LLM implement those gaps.
The key thing here is that, by doing this, I DO know how the code works. I may not know how each unit is implemented, but I know each unit’s responsibilities and how they interact with one another. I’ve found that LLMs are HORRIBLE at writing decent abstractions, regardless of how much prompting and skills I’ve given it. It will always get the job done, but it will often be unmaintainable and require bigger and heavier refactors as I introduce more and more requirement changes.
But by flexing my basic design skills, I don’t have those issues. Future refactoring fit into easy abstractions that I’ve designed. And by documenting it up front, I also have what becomes a living document on how this work will turn out.
It takes a few hours up front to plan ahead, but it makes the LLMs job easier and use less tokens, and it makes the code better long term. I don’t care if one unit is suboptimal, as long as the abstraction is right — because at worst I can change an implementation for a better one later.
forestry 2 hours ago
This is where myself and a colleague have settled. We design for longevity but it can still type, search, and debug faster than we can.
mentos 2 hours ago
Same. I let the LLM work inside a framework that I made.
Of course I only arrived at the framework after trial and error with an LLM showed me the shape of the problem and the framework presented itself to me.
apinstein 2 hours ago
Same here. I review plans w rendered diagrams. Data structures, flowcharts, uml sequences, and component architecture docs.
I have found that making the AI look the plantuml and the rendering and reviewing before presenting it to me results in much more legible designs. To me the peak of this would be a shared interactive whiteboard w the ai during design.
smashed 2 hours ago
Agreed. Even with opus 5.5 which everyone praises for its wonderful coding abilities, I keep catching awful coding abstractions or missing abstractions.
It can keep patching everything left and right whenever a new requirement comes up so it does not "need" to build abstractions, and it will only do it to please the human reviewer.
j45 2 hours ago
Agreed, otherwise the risk is receiving the average of what LLMs think are best practice.
Planning is fundamentally so critical to ensuring alignment and outcomes, that over time how one plans can be better trained into the process to help do more and more of it.
roadside_picnic 2 hours ago
> I have created.
Are you working on a team? I've found that for solo projects this works but, even when you own the design of your piece, integrating with the work of several people doing this ultimately leads to even the part you initially designed no longer working exactly as you remember it.
A good chunk of my time now is spend working with an LLM to understand how other parts of the code base work and how they relate to and interact with the product features I manage. But I'm still on a small enough team that this is possible. I'm not surprised at all that an engineer at an org the size of Meta would not even be able to perform this level of work in understanding what's happening.
bitwizeshift 18 minutes ago
Yes this is on a team that I lead. We have augmented our practice so that teams do the design reviews together to ensure we are all understanding the changes. It’s also helped uplevel devs to think in terms of design practices.
It still has sharp edges but IMO it’s the best we have at the moment, and it works. It also means we deliver larger volumes of code, but we mostly look for alignment with the original review — so our reviews are more focused on what tests say behavior should be
eggplantemoji69 2 hours ago
This is too the paradigm I’ve settled upon after a year of using these tools.
fasterik 2 hours ago
I agree with this approach, but I'm a bit surprised this has to be said at all. These are basic software design principles and I imagine it's the default way any experienced programmer would approach the problem. I wonder if the people not using this approach are primarily junior developers who have never seen how real software is built, or if it's leaking into the senior levels as well.
swatcoder an hour ago
Most people in the industry right now, even at FAANG and even among people carrying "senior" titles at these places, have little experience with "software design principles" -- tens of thousands of brilliant Ivy League graduates with surreal compensation packages have spent the last decade pulling tickets off JIRA and pasting least-change-possible patches onto existing code for about 10 hours a week while spending the other 30+ hours barely engaged at meetings.
JohnMakin an hour ago
This hurt to read but checks out in my experience. These types also tend to love tossing debt over silo'd walls and pointing the finger a lot if/when anything goes wrong.
I'm old enough now to be a protected class, so I don't want people accusing me of ageism, because that's not where this is coming from - but there's a type of person that clearly is on their last job of their career and knows it and I try to avoid them like the plague, I've seen it too many times.
trio8453 an hour ago
The LLMs know design principles too, being an experienced developer doesn't make you as special as you might think.
fasterik an hour ago
Don't get me wrong, LLMs are way better than me at most things, but I don't think you can deny that the current generation of models makes questionable decisions sometimes. Especially on project structure, API boundaries, and code duplication. I also think they just have bad taste in general, though of course that's more subjective.
trio8453 an hour ago
> I don't think you can deny that the current generation of models makes questionable decisions sometimes
Sometimes, not enough to believe you that you can do it better. And even if you could, to believe that it's overall worth it to work in that way.
fasterik an hour ago
You can work the way you want to work. In my experience, letting the agent make every decision leads to a big ball of mud that requires touching dozens of files to make a change. More context is needed, it burns through tokens much faster, and the agent takes shortcuts and builds technical debt. So the way I prefer to work is to have changes targeted to a single file or small set of files where I've defined the API. Maybe that will change in the future, I'm not set in stone here.
zmmmmm a minute ago
> In my experience, letting the agent make every decision leads to a big ball of mud that requires touching dozens of files to make a change
It seems too early to make any conclusion like that. We've still only been doing agentic coding for a year and capability significantly improved during that time. I don't think anybody on either side of this debate can have actual empirical evidence to support conclusions about where it lands longer term.
The main argument I see is that it probably pays to be slightly conservative because the cost of being wrong by maintaining too much oversight is better than the cost of shipping slightly less code. In some cases it will be more than "slightly less" but in almost all complex pre-existing systems, you are rate limited by a lot of other factors outside of code production, so the effective difference just isn't that high.
jonplackett 43 minutes ago
What still is true for all fields of LLMs: if you can’t tell they do it suboptimally, you are just not that knowledgable in that area.
It still applies to all things LLms can do from law to code to art to ideas.
They can do it - and very acceptably - but only to the untrained eye.
tokioyoyo an hour ago
I’m not sure how to say this without sounding smug, but the last year has shown that most of those things just… don’t matter. It’s just the next level of the “variable naming PR comment battle”, so will become obsolete very soon.
Obviously there are mission critical systems where it matters, but if “your code can be A/B tested” or “slowly rolled out”, chances are, it doesn’t matter.
swatcoder 39 minutes ago
You'll find that design matters when token spend starts facing real accounting.
Tech debt is still tech debt, even if there exist models that can wade through, because it's going to require increasingly sophisticated models navigating increasingly large context windows and increasingly speculative workspace operations to keep making progress on it.
Vendors like Anthropic and OpenAI are laying a pit trap when they demonstrate how their models can independently code their whole platform as it becomes increasingly byzantine and inscrutable because not only do they benefit from internal/prerelease models but they have effectively unlimited spend because all that spending acts as a lure to draw you into their trap.
For everybody else, design still matters. People are just working really hard to make you forget.
birdsongs an hour ago
> doesn't make you as special as you might think.
Way to come off as incredibly insecure. There's no need to randomly put people down.
trio8453 an hour ago
I'm not the one who's claiming to be above the latest models in some aspect of coding in this interaction.
birdsongs an hour ago
No one at all claimed that, certainly not the person you responded to. They're talking about how to use these tools in a way that gives you insight into what they produce. So, as an engineer, you know the shape of the output, before it's made.
That's it. Maybe you should go re-read?
alfjal 14 minutes ago
Why is that so hard to believe? I don't always do this, but I do course-correct Claude on almost every big change. YOLOing it would superficially look fine, until the latent issues catch up with you.
fluoridation an hour ago
Talking about LLMs "knowing" things is questionable at best.
chasd00 an hour ago
I've been doing this a long time and i would say since the last 3 months or so i've spent more time learning from llm code than correcting llm code. It's not bad and takes approaches and makes decisions i wouldn't have thought of. I'm not too ashamed to admit that, a model just as access wider and deeper experience with code than I do.
hathawsh 35 minutes ago
The LLMs have been trained on all kinds of code. They often choose a hacky solution because that's what they encountered in similar situations in their training. It generally only takes a few words to guide them to a better design than what they saw in the wild, but if you don't guide them, they follow the herd.
birdsongs an hour ago
I work on my code this way, but I have no idea what my colleagues are doing, and it's impossible to review thousands of new/changed lines a week.
I just approve PRs, because I have to. I don't know what else to do. I can barely keep an eye on all my code.
Actively looking, but job market is rough in Northern Europe. I quit caring about quality in any capacity here, the company has clearly chosen its direction, against my very vocal complaints.
My last interview, with what I thought was a great company, failed when I didn't understand what NLP was when they asked me about it.
> NLP?
> Natural Language Programming, it's the future. Claude, GPT, etc.
I laughed, he didn't, looked offended and said it was a serious question. I had thought it was a joke, said "oh you mean LLMs". I tried to answer seriously, of course, but that was my third round technical interview and I didn't get a call back.
Just getting a paycheck and producing mediocre work until I get something new. (Senior engineer, 10 YOE, patents, 3 years at a unicorn).
Also considering becoming an electrician.
fasterik an hour ago
Yeah, I imagine it's something that needs to be enforced at the organizational level. I sympathize with anyone who takes pride in their work and needs to deal with mindless coworkers pushing slop without any accountability. I left corporate software development (before LLMs were a thing) over frustration with the general culture and I can only imagine it's gotten worse since then.
birdsongs an hour ago
It's really bad now, yeah. :( It just really hurts that I spent well over a decade of my life learning and honing skills to engineer safe, quality software and now it's just, useless.
No one cares at best, or I'm callously told "keep up or fall behind" at worst. When that's not even the problem, it just, sucks now. There's no joy in anything I do professionally.
Can I ask where you went? I'm seriously looking for an exit, I just don't know what yet. Maybe a trade, in Scandinavia the pay is actually comparable, I just don't want to go back to school.
fasterik an hour ago
I was fortunate and saved up enough to live frugally and just work on passion/learning projects for a while. I've been considering going back into commercial software development as a solo dev and leveraging AI, but keeping the focus on quality and craftsmanship. In my opinion there's never been a better time to start a new, ambitious project if you're passionate about quality software.
Silhouette 6 minutes ago
It just really hurts that I spent well over a decade of my life learning and honing skills to engineer safe, quality software and now it's just, useless.
It can feel that way with all the hype but I very much doubt that is really true. I see an industry that is rapidly dividing into two camps - "move fast and break things" and "this actually needs to work". Writing everything with agents and merging while barely even checking what is in the PR has become standard practice in the former but it will get you laughed out of the room in the latter.
fluoridation an hour ago
Sounds like a cushy job. Why would you want to leave?
swatcoder an hour ago
Some people just can't turn off their brains and play drone for dozens of hours a week for decades. It feels existentially, aesthetically, and sometimes ethically crushing to spend a whole life doing work you can't expect yourself to respect or value, no matter how much you get paid or how little the work overtly demands of you.
Often, the lack of meaningfully challenging demands being put on you is punishing on its own. It can feel like a relief for a week or month or year, but if you catch a glimpse of that being your whole future, you can come to recognize yourself as being in a mind-eroding secular purgatory.
This anguish is especially common among on highly capable people, high achievers, and those used to some kind of ambition. And by accident of its history, there's a lot of those people in this industry and some of them are going nuts now.
fluoridation an hour ago
Why not become a doctor in that case?
Nevermark 43 minutes ago
> This anguish is especially common among on highly capable people, high achievers, and those used to some kind of ambition. And by accident of its history, there's a lot of those people in this industry and some of them are going nuts now.
I went nuts for years, until I sold off my interest in my dream partnership.
10 years of heaven. All the benefits and none of the downsides of my internal colleagues. Then 14 years of bureaucracy aimed at predictability over progress, which was infinitely more difficult for me to dodge as an external player.
Compensation for my interest was very objectively valued, with the rate of stalled progress priced in. Which made complete sense, while also being its own madness.
I walked away from so much value, but I don't regret it for a minute.
birdsongs 29 minutes ago
> and some of them are going nuts now.
Pretty much this. I know it sounds melodramatic but it literally feels like rotting.
I don't solve anything anymore. I don't get any spark of fixing bugs. I don't DO ANYTHING, and it's fucking chipping away at me.
Also I'm fairly sure people are going to realise soon that if they're happy with slop, they don't actually need highly trained engineers anymore.
This whole thread actually prompted me to look into electrician work, especially in renewables that we have here in Norway.
I might fix my burnout and go to school to get my 2 year qualification. AI can't rewire a hydro dam.
fluoridation 19 minutes ago
You should instead phone it in and use the easy paycheck to learn something you're interested in. I phoned it in for years in the crypto industry.
tomat_1000 17 minutes ago
Man, I have the same feeling.
A long time in your life you build the career and now it’s just feeling so empty, like you are not a engineer anymore, now everyone can use AI and without checking code they would have good results.
hathawsh 44 minutes ago
I will be unhappy if they steal the meaning of NLP. NLP means Natural Language Processing and LLMs are only a subset of NLP. I would have concerns about anyone who claims to have technical knowledge and asserts NLP means "natural language programming." Those are the same kind of people who might suggest Lisp should add mutable objects.
riffraff 36 minutes ago
You may be surprised and annoyed that NLP is also "Neuro linguistic programming", I've met technical people talking of NLP and meaning that...
hathawsh 24 minutes ago
Yes, but that's in psychology. Different fields have their own terms without causing too many issues. Communication breaks down when we reuse terms for different things within the same field.
arcanemachiner 22 minutes ago
More like parapsychology...
d1algo12 an hour ago
LLMs certainly helps with designing and is the best brainstorming tool one can have beside their own brains.
easterncalculus an hour ago
> rather than let the LLM do the entire structure, I take the time to create the full hierarchy of abstractions and actors that I want my code to do. I decide, at the UML level, how abstractions will sit and interact — and then I make the LLM implement those gaps.
Do you do this with every repository that's brand-new to you? You refuse to, for example, use an agent to learn where the feature should generally live? This seems absurdly slow for that purpose, because even though humans can still get a deeper understanding of a codebase after several months, an agent can get a much deeper understanding over a timeline of the initial few days. Unless everything is already meticulously documented for you, I suppose.
bitwizeshift 12 minutes ago
I don’t believe I said I don’t use agents for things like research? There’s a difference between trying to get your bearings in a new codebase with an LLM pointing you in the right direction so that you can understand things, and just letting an LLM like Claude take the wheel to do a change for you.
Yes, new codebase we take the time to learn, as every good developer should. Ramp-up doesn’t have to be that long to get one’s bearings.
Also, changes don’t have to be deadbolted into bad designs. If the code was bad before and you have a change to make, it can sometimes be faster in the long run to redesign it and rebuild it so that it can scale better with future requirements. LLMs make this part fast and easy — but again, requires design, and designs require human brain involvement and understanding of the codebase
My conclusion was I had to elevate myself and make sure another agent reviews and tests the code (QA) under the supervision of a third agent (CTO) that defends my vision.
ChrisMarshallNY 3 hours ago
Testing.
Lots and lots of testing. I usually interactively test with the LLM. Now that I'm spending less time, coding, I get to spend more time, testing.
In my experience, I test more, with an LLM, than I did, with my own code. I don't find coding errors, but I do find implementation errors.
On another note: I ran an LLM on a bunch of really mature code that I wrote, and could swear was "perfect," and it found a number of issues.
Nobody's perfect. Trust; but verify.
SimianSci 3 hours ago
So, in other words, we save time and money when it came to creating stuff, but we've made a whole bunch of new work around verification and testing of said stuff?
What have we gained aside from shifting where time and effort is spent in the design loop?
thevinter 3 hours ago
Better tested software duh
theK 3 hours ago
Isn't this the perfect classical capitalist advertisement? Every number goes up.
- code? up
- build budget? Up
- developer output? Up
- developer oprrational costs? Waaay up!
Why is this Bad? Every one involved profits somehow.
alpha21io 2 hours ago
Shipping features and fixing bugs much, much faster.
onion2k 2 hours ago
we've made a whole bunch of new work around verification and testing of said stuff?
The verification is what we always wanted but never had time for. For 30 years I wrote side project games without extensive testing or a fuzzer or anything because I wanted to spend my time writing the game. Now I can do that and still have the testing I always knew I should be building.
ChrisMarshallNY an hour ago
Engineering will always be difficult.
Right now, we're in a "honeymoon" stage, where non-engineers think that they can somehow, magically produce stuff that is comparable to what experienced and disciplined engineers can do.
That won't last long.
Despite all the caterwaulin', LLMs are just another engineering tool, and good engineers will figure out how to implement them, in ways that absolutely blow away the garbage produced by non-engineers.
It's like that rich, no-talent guy that buys a PRS guitar, and thinks he'll beat the experienced musician that still uses his beat-up old strat, but consistently makes hits.
kreneskyp 32 minutes ago
I'm largely building tools for myself and my agents. This allows me to battle test internally before releasing code to the wild. It adds some time but largely it's part of my regular work unless something breaks badly. (I also run multiple agent teams simultaneously and do staged rollouts to reduce blast radius of the latter)
This method also helps with the classic problem of being unable to identify essential complexity up front.
michaelt 3 hours ago
I find the problem with "lots and lots of testing" as a solution - (assuming by that you mean lots and lots of automated testing) is it invites lots and lots of lines of code - making code reviews even more challenging.
If someone finds it burdensome to review my 50-line AI-generated bug fix, adding 200 lines of tests only gives them more lines to review.
AnotherGoodName 2 hours ago
Might be a little controversial to say this but automated tests can be reviewed at a lower bar than production code.
I have no shame in accepting vast quantities of ai generated automated tests at review time without really paying attention to them. It’s better to spend more time on the core implementation.
Silhouette 2 hours ago
I have no shame in accepting vast quantities of ai generated automated tests at review time without really paying attention to them.
I respectfully disagree. IME writing a good test suite is one of the areas where current LLMs perform worst. They certainly like to write plenty of tests but those tests aren't necessarily adding much value and they don't necessarily cover the important functionality.
Given how heavily LLMs and specifically agentic workflows still rely on producing something vaguely correct and then iteratively applying guard rails to reach something that actually works I humbly suggest that trusting a very substandard test suite to be one of those guard rails might be unwise.
flir an hour ago
IMO it works better if you split the tests into the ones you care about, and the ones the LLM writes. I assume its tests help it write code with fewer errors, so I'm happy for it to keep doing that, but they're akin to CoT - an artifact of the process that I don't need to inspect in detail.
I realise this is heresy, but my conclusion is that unit tests aren't important any more. E2E testing on the other hand...
(I'm agreeing with you about the quality of LLM tests, I'm just trying to make lemonade out of all these LLMons).
Silhouette 17 minutes ago
I admire your optimistic approach. :) Unfortunately I don't really share your optimism on this one.
All those extra tests are churning through tokens and CI resources. These are far from free in a direct financial sense. I've also noticed some of the more recent models will try to do a lot of experimentation and ad-hoc testing by spinning up shells or running something like Python scripts direct from a command line. Every time they do that you either have to review whatever they propose to run carefully - which is slow and tedious work - or stick the tools in YOLO mode and hope that your agents don't accidentally wipe your entire filesystem, open your firewall for malware to take over your network, and perhaps launch a few nuclear missiles.
I would respectfully disagree about the value of unit tests as well. The reason unit tests appear to have less value in the LLM/agent era is because current LLMs suck at decomposition and other architecture-level work. They just see a mass of code and change it as a mass of code. This probably results in orders of magnitude more code being written/changed than would have been necessary for a skilled developer working on a well-maintained code base to solve the original problem and that automated work also takes longer in wall time. Again this gets brushed under the carpet because a lot of people trust that LLMs are so good that their costs are negligible regardless of how much work they do. People used to say that about the Cloud transition as well because not needing so many in-house staff to manage your resources was going to be a huge saving. Today we have companies bringing in consultants because they literally can't even work out how much they're spending on AWS by themselves. Instead of having a few greybeards who could make a Linux server do anything we have a few increasinglygreybeards who know which combination of 17 AWS resources they need Terraform to spin up so they can write the web server equivalent of hello-world.c. Meet the new boss - same as the old boss. :)
ThrowawayR2 2 minutes ago
[delayed]
AstroBen 2 hours ago
How do you test code that you don't understand?
Black-box e2e testing isn't suitable, or practical, for a lot of applications. Unit and integration tests generally require you to understand the code you want tests for, to know what to actually assert
Are you just trusting the AI to do it all?
avgDev 2 hours ago
Nothing a "make no mistakes" at the end of a query cannot fix. /s
g-b-r 2 hours ago
The point is that you're not verifying if you don't even review the code.
Even if your test suite tested every single possible behavior and code path, tests usually can't intercept all possible unintended behavior that an LLM might have added.
In all likelihood, most test suites don't intercept any unintented behavior (such as unintended internet connections).
That's not a big problem when a honest human has written or reviewed the code, because unintended code is usually easy to spot.
But an LLM might have dropped any whatsoever code: maybe some of your project's code resembles the one of a trojan in its training, so it completes it with the trojan.
Or maybe it found that if your server is unreachable, it can upload the client's data to a public Github repo; or that if there's no space left on the device, it can delete some of the user's folders, easy!
Would your clients be understanding if that happened?
ChrisMarshallNY 2 hours ago
So you currently routinely review the machine code and link tables produced by your compiler?
Eventually, the tools become trustworthy, and we no longer need to worry about their products.
We're just about there, with frontier LLMs, but I don't think we're all the way there, yet.
AstroBen an hour ago
"I saw her duck" - how much would you be willing to bet on the intended meaning of this sentence?
English isn't a good programming language if you want to trust that the output will be as you want it to be. High level code has no room for ambiguity - that's why we use it. It's a concise, very explicit language to tell the compiler exactly what to do.
Dealing with an LLM is a fundamentally different thing. The reason it's able to speed us up is because it makes assumptions for us and reads between the lines. The only world where we trust the output is one where it can literally read our minds. That's not coming any time soon
g-b-r 40 minutes ago
How can a software developer compare a deterministic program with LLMs?? Please tell me that you aren't, or that it's not what really think!!!
onion2k 2 hours ago
I recently moved my project guardrails into a separate set of skills in a repo that I can bring into a new project fairly easily. It's focused (wrong word, it's seriously not focused) on implemented as many different levels of testing as I can think of. Right now it does this https://github.com/onion2k/guardrails/blob/main/GUARDRAILS.m....
The goal is to get to a point where I can say with a good level of confidence that a change didn't break anything. It might be the wrong change, and it might be crap code, but I can ship it without anything blowing up. I've been using it for a while across a few game projects and so far it's worked quite well.
adamddev1 an hour ago
> Trust, but verify.
Testing is NOT verification.
EDIT: Sorry, oops, it's also called a kind of verification.
yjftsjthsd-h an hour ago
I'm pretty sure it is? It's not validation, but what are tests if not automated verification of certain things?
adamddev1 an hour ago
Sorry I was thinking like formal verification like proving something is correct. I guess testing can be considered a form of "dynamic verification."
But there is the problem that tests only check or verify a small sample of possible cases or inputs. They can never show that the program is correct. (Famous Dijkstra statement.) To know something is really correct you have to understand and prove it.
senderista 33 minutes ago
Testing doesn't verify the design.
mbesto 28 minutes ago
On this note - people love to create 'AI factories'. Has anyone made a "QA factory" where a set of agents mimic real users to run in a pre-pod environment? Like I'm talking you spin up a variety of VMs with real desktops and agents that use real browsers and act like real users (on timing schedules that mimic real users too)
spacedcowboy 11 hours ago
I’m slowly coming around to the idea that if it works, I’m good with that. I never used to personally check all my colleagues work, either, I just trusted that they’d done a good job.
That’s not to say I blindly trust an LLM - my project is an autovectorising compiler. I have it write a bazillion tests, and I have real, non-trivial programs that it compiled. I have it run a ‘gate’ before releases that takes literally hours on 5 different architectures. To say this code is tested is … masterful understatement.
I think what helps is a long planning session, going into details, working out corner cases, designing test-cases, you know work and effort. Once that’s done it’s easier to let it work through the night and do a major production feature.
I still review the code, but hand on heart I couldn’t say I’d reviewed everything, sometimes I miss something in a complicated check-in. I try to keep checkins small and contained, but sometimes that’s not easy to manage alongside the “don’t break stuff” rule as well.
So it comes down to: I’ve been working on this for 6 months or so, and so far this has worked out for me. You know, kind of like getting used to a colleague’s idiosyncrasies over a period of time.
I will say that in my experience the code quality is fairly high. Maybe that’s down to the effort put in ahead of time in the planning stages, I don’t know, but I don’t seem to suffer from the ‘slop’ that people complain of. YMMV.
yulaow 10 hours ago
I am not in META but for what I read on blind, assuming it is true, they are not spending any time in long planning sessions or in getting into details of the implementation plan at all, and they could not anymore considering how many PRs in parallel they are requested to manage
spacedcowboy 10 hours ago
Yeah, I'm retired. The only person I have to answer to regarding progress is me. I'm a stubborn son of a bitch when it comes to choosing quality over speed...
That doesn't mean I don't like progress mind :) This morning I got the preliminary results of a 'C' front-end to the compiler (it uses clang to compile to LLVM IR, then translates to our IR, then feeds that IR into the autovectoriser as if it had parsed the code itself). So for the exact same source code I now have:
Those are some pretty fine speedups, from naive C, with no alterations from the standard "C" code - it automatically finds places that can be made data-parallel, tracks the data-hazards to make sure its safe to "GPU" and writes all the kernels, boilerplate and data-stream-management if necessary. Obviously the checksums have to match, so I know it did the work correctly too :)
Hacker News isn't interested - any post I make linking to the webpages is immediately blocked/dead so I'll wait until the C++ front-end is ready (some time next week) and announce it somewhere else. A compiler that takes standard C/C++ code and speeds it up by automatically finding data-parallel areas and scheduling them on the GPU with no programming overhead is pretty rare.
Oh, and it manages the local network as well, so you can run xcd on machines on your lan, or via the TB4/5 bridge between Macs and it'll schedule on other GPUs, just like it manages multiple GPUs on the local machine. It characterises both latency and compute-time to figure out which devices are worth scheduling on.
ch_sm 3 hours ago
That is pretty adventurous, but it sounds really cool! Will it be open source?
spacedcowboy 2 hours ago
Yep. I’d give the link but I think HN thinks I’m spamming because I’ve mentioned it too many times before in posts - hence immediate death of any submission, so I’m not mentioning it for a while to see if that tails off.
Bottom line: GPL3 for the compiler, standard library exclusions for derived works, so use it for anything but feed me back improvements to the compiler itself, and you don’t get to redistribute it as your own.
jazzyjackson 3 hours ago
Hell yea I love that computers are getting another chance at becoming compute fabric, just using each others GPUs. Sick.
JMKH42 3 hours ago
I like it!
The Rust crew would love this I bet.
rpdillon 2 hours ago
Awesome, this is genuinely-cool HN material. Thanks for sharing.
Silhouette 2 hours ago
Upvotes don't appear publicly on HN so I'll comment to agree with this. I'm sorry if earlier information about this compiler got lost in the whirlwind. Data parallelism is a hard problem and these results look genuinely impressive so I for one would definitely be interested in hearing more about it and maybe discussing it further when there are interesting developments.
spacedcowboy 2 hours ago
Thanks :)
Still working on some of the data-parallel recognition in LLVM IR space. In order to support both C and C++ with the same data-parallel section extraction, I moved the detection from the ‘xc source code’ phase to the IR lowering phase, meaning that everything could ‘feed in’ to the same routines. That’s caused a little friction with the slightly different ways the LLVM IR from clang and xc’s IR encode loops. The above results are all single-loop extractions and the double-loop ones are still ongoing.
The system works in 3 stages: 1) by being greedy about which loops could possibly vectorise and marking them, 2) by analysing the IR to reject loops which do pointer-aliasing, or object manipulation, or… and then 3) by measuring the fastest way these known-safe kernels could execute.
Which brings me onto one nice thing I haven’t mentioned: that the GPU choice(s) are made at runtime, not compile time, and the system automatically measures (and then persists so it doesn’t have to measure again) the fastest way to run a kernel on the current machine. It handles MSL on Apple devices, PTX on Nvidia cards, Spir-v on Windows, Linux, and Android, and WGSL in the browser. It’ll quite happily divide up the parallel task across a 3090 and a 5080 on my Windows test box for example, with the 5080 doing ~2/3 of the work, because the scheduler is opportunistic.
All without writing anything other than plain C with nary a mention of the GPU. Give me a week or two, and I’ll probably announce it on phoronix (which might then get a mention on HN if it goes down well there, and it’ll be a different url so hopefully won’t be insta-dead).
bee_rider 2 hours ago
So, the real input to your autovectorizer is LLVM IR; you could soon enough support any language that has an LLVM front-end? (I say soon enough instead of immediately because I’m sure there’s some manual wiring up to do). That seems like a pretty big deal. Do Fortran next, pls. :)
spacedcowboy 2 hours ago
Ssshhh. I like to try things before promising them :)
smokel 2 hours ago
> Hacker News isn't interested
I wouldn't be too sure about that :)
spacedcowboy 2 hours ago
I’m being a little cynical - I’ve submitted it twice (though the ‘C’ front end is new) and it bombed both times - the second time within seconds of submission.
I think it’s my own fault, I’ve posted the link to the compiler pages several times in what I thought were relevant posts, and I think HN thinks I’m spamming. So I’m not posting the home page for a while and we’ll see if a future submission “takes”
john_strinlai 2 hours ago
might be worth firing an email to [email protected]. they should be able to tell you why your submission was auto-killed, and might work with you to ensure your next submission isnt.
spacedcowboy an hour ago
I actually did email the second time because it was clearly an automated response, but I didn’t hear anything back. Not too surprising, Dan is a very busy guy, and he doesn’t know me from Adam :)
ggsp an hour ago
After asking my LLM some questions to barely grasp what you are working on, I can now say: very cool!
sounds very cool - is this basically "C" support for cuTile? :)
chrisjj 3 hours ago
> I’m slowly coming around to the idea that if it works, I’m good with that.
A.k.a. "The concrete didn't crumble when I hit it with a hammer."
Process quality has been a thing for ages because it is essential.
spacedcowboy an hour ago
I mean, honestly ? Yeah.
Except it’s about 1000 “hammers” (aka tests), x 9 different architectures, x 2 different (a reference and a production) variants, and x 3 different optimisation levels. Every time I find something that slipped through, it gets at least one test written to cover it in the future. My main time-sink is the hours it takes to run the ‘gate’ before a release across the test-farm. And that’s after I’ve been playing with it myself for a while, pushing corner-cases through whatever I’ve been working on.
jimbokun 2 hours ago
Yes, compilers are a good candidate for verification through thorough testing. If a comprehensive test suite passes, reviewing every line of code isn’t so important.
If your definition of "works" includes "doesn't have vulnerabilities", you're going to need to do more than just poke at it and add tests.
MisterMunchkin 25 minutes ago
How does writing the code by hand solve that? If anything it’s less likely to be secure, because you don’t contain the sum of all human knowledge.
sixothree 2 hours ago
Very similar here. For internal projects the test is - if it works, unit and integration tests were created, and it follows our architecture correctly.
dmitrygr an hour ago
> never used to personally check all my colleagues work, either,
Jesus F Christ! No wonder most software lately (~decade) has been shit. Have y'all not heard of code review? Or did you think it just for show...you sit on a change for a week and then press "Accept" without reading it?
spacedcowboy 2 minutes ago
My dear fellow, the statement was that I never used to personally check all my colleagues work. Of course I took part in code review (but then, that’s pretty much the same as me reviewing the LLM code, no ?) The statement is that I personally did not review every line of code in the repo. Some of it was authored, reviewed and committed by others.
FWIW I was a software engineer for the last 35 years or so. Amongst other claims to fame, pretty much all Apple devices on the planet (those actively running Apple OS’s, anyway) are running my code, within AuthKit in fact, and one code review was insufficient by policy when making changes. Despite all that, I’d be lying if I told you I thought there were 0 bugs in the actually-rather-complicated repo… Maybe these days, running it by an LLM would be a good idea…
4ndrewl 3 hours ago
This is how every product manager feels every day.
If you're an LLM prompter shipping products, you're outputs are more akin to a product managers, not an software engineer's.
techscruggs 3 hours ago
Agreed. The implication is, get really good at defining acceptance criterion.
mikeocool 3 hours ago
Kind of, except as product manager you're held accountable for delivering the right product, while your engineering team is held accountable for things like performance, incidents, and security issues caused by their code.
It's hard to hold an LLM accountable for anything.
theK 3 hours ago
Accurate but manageable. Just like a pm works with properties an engineer also does. The Product person wants to deliver the right product properties so they model their process around identifying the ones they want and then creating the right framework to get them. On the engineering side its the same thing. Define what you want, give agents an environment they can succeed in, iterate.
i_love_retros 3 hours ago
What nonsense! If a product manager ships something that breaks then the software engineer who submitted the code gets the blame!
Software engineers are always responsible for their code except now they have to be responsible for Claude's code
4ndrewl 3 hours ago
Oh, don't worry. Someone is also blaming the product manager!
Silhouette 3 hours ago
Software engineers are always responsible for their code except now they have to be responsible for Claude's code
Which is - of course - absurd. Accepting responsibility without demanding the corresponding authority is a fool's game.
I have never been much of an advocate for unionisation in the software development industry. I didn't think it was necessary to protect the interests of the workers because for most of the industry's history the workers have largely been able to stand up for themselves anyway. Likewise I've never been a big believer in accreditation and licensing practitioners in our field - because who is qualified to judge someone else worthy in such a young industry with so little hard data about what works or doesn't work? I find my views on these subjects are evolving now.
It's not about protecting jobs if they truly aren't needed any more or protecting people working in the industry who refuse to move with the times. The Luddite philosophy isn't going to get us anywhere here. And besides - if AI is as effective as its advocates have been claiming then there should soon be more demand for capable developers to take advantage of the new opportunities. (The Jevons Paradox is relevant here.) But it is about protecting standards. If the tools aren't doing a good enough job then expecting professional developers to ship the code produced by those tools even though they know it's junk is something that needs to be challenged. And if developers are being expected to ship anything important without having the slightest idea whether it even works then that's even worse.
We had a lot of silliness going on when the very low interest rates and huge amounts of VC funding were propping up ideas and practices that would never have survived otherwise. Today we have a different kind of funny money distorting the industry but it's still propping up ideas and practices that we wouldn't normally support or perhaps even tolerate. And just like before no individual is going to be able to hold back the tide and it's going to have to be a collective effort to push back - hence my evolving views on topics like unionisation and accreditation. We've all seen the drop in quality across much of the industry and we all know why it's really happening because we see how the sausage is now being made.
p2detar 2 hours ago
Not all mistakes are code, a lot are feature design mistakes, and if SWEs would always be responsible about PM mistakes, then why not use Claude anyway?
Creamsicle47 9 minutes ago
It comes down to the blast radius of a failure happening in your system. Some stateless frontend won't require much beyond visual inspection, but if you introduce some client-server architecture or require payment processing, you must consider the cost of a security intrusion when auditing code changes.
jeffrallen 3 minutes ago
[delayed]
solatic 3 hours ago
It's not too dissimilar to life as a Principal level engineer pre-AI. You technically oversee the work of many engineers on their own systems, but you couldn't possibly review all the code yourself. You appreciate that human code review reduces, but does not eliminate, the number and severity of bugs shipped to production. You don't have the low-level details in your head, just the high-level ones. You design how all these engineers interact with each other, without stepping on any toes.
Time will tell whether we were right to move from a system where we trusted humans to understand the low-level system, to one where we trust fancy calculators. But so far, it's working.
akomtu 3 hours ago
The assumption was that some engineers knew how each part of the system worked, or at least the system was comprehensible because it was created by humans for other humans. There were comments explaining difficult places, etc. With AI this assumption is broken. Now nobody understands how things work, and it cannot be understood in principle because the system is a mountain of slop created by some alien intelligence.
solatic an hour ago
> Now nobody understands how things work
Well, the fancy calculators do. Yes, they still need guidance to keep from going off the rails. But, at least from personal anecdotal experience, I need to go spelunking into the codebase less and less over time. The agents and models really did just start to become that good.
Silhouette 2 hours ago
But so far, it's working.
There seems to be disturbingly little objective evidence of that. There's certainly a lot of code being written but how much actual value that code is adding is another question. With the scale of resources that have been pumped into AI-supported software development over the past few years where are all the new applications and the quantum leaps in quality and performance in the applications we already had? If anything I seem to be seeing more anecdotes going the opposite way so far.
solatic an hour ago
Anthropic's recent blog post about how they replaced Git should be a massive wakeup call. A trillion-ish dollar company (exact number unimportant here), the vast majority of its value coming from its intellectual property, replacing the storage layer that is an essential part of its system of record for that intellectual property, written pretty much entirely with its in-house Kool-Aid.
Disturbingly little evidence? I would hardly call that "disturbingly little"...
Silhouette 31 minutes ago
(Apologies if I'm misreading your reply. I'm assuming it was sarcasm but Poe's Law might apply here.)
This does feel a lot like the "Agile era" that was the last silver bullet that was supposed to fix software development forever and the "Cloud era" that was supposed to fix infrastructure and deployments.
Agile had a few good ideas. Code reviews and automated tests are commonly used today.
It had a few plausible ideas that it took too far. We are better at recognising the value of incremental development over big up-front design today - despite the ironic clash this seems to be having with agentic workflows - but almost everyone hates stand-ups and lots of projects have gone off the rails by breaking things down far too much and never committing to anything and so never getting anywhere.
And it had plenty of ideas that almost all of us have thrown out as unpleasant and/or unworkable. Pair programming comes to mind immediately.
What didn't really change was the fundamental nature of software development. Did we adopt a few useful ideas and get a more more efficient and improve quality a bit? Certainly. Did we get orders of magnitude better at anything and give up all the tried and tested ways of working in favour of radical new ones that never quite felt right? No.
On our current trajectory I'm expecting a broadly similar outcome for LLMs. They're now very good at some things. But they are still pretty bad at other things that also matter - and some of the reasons they are bad at them are an inherent part of how they work and therefore probably can't be overcome (at least not with incremental improvements on the technology that still basically work the same way). So I am expecting to see a period of consolidation with a lot of organisations pulling back from their dramatic leaps into the unknown and downsizing of dev teams once the big AI companies start squeezing financially and the problems of developer burnout and stagnating skills and low morale that we're already seeing after just a few months become endemic. And then as I suggested in another comment I think it's also increasingly possible that we'll see developers unionise and/or push for professional regulation and licensing so they can resist the pressure to just ship whatever their agent wrote without even bothering to check it but still being responsible when inevitably things then go wrong.
skeledrew 2 hours ago
Technically it's just a role change. You're now the project manager or something, not the developer. It isn't a project manager's job to understand the code, only to ensure it does what it should and doesn't do what it shouldn't. And to ensure the developers, etc str unblocked and doing well. Embrace the new role.
gonzalohm 2 hours ago
Why would you need to embrace something forced upon you?
jimnotgym 2 hours ago
Perhaps it will make it easier to cope with? Fighting something beyond your control is pretty exhausting
gonzalohm an hour ago
Or just ignore AI. As long as you do your job, who cares if you use AI or not
skeledrew 2 hours ago
In most workplaces, when supervisor/boss says "we need you working over here now; you start tomorrow", you either embrace or quit. Again, nothing fundamentally different.
gonzalohm an hour ago
And in most jobs, you don't get promoted to manager if you don't want to
skeledrew 42 minutes ago
Maybe not, but things and times change. If you're made obsolete in one role then you better be ready to move on to another.
episteme 2 hours ago
So you can pay rent?
testplzignore 2 hours ago
There is no escape from Dario's basement. It's best just to accept our fate. Rub the lotion on our skin or else we get the hose again.
gonzalohm an hour ago
We just need to hold until people realize the crappy codebases that AI is generating
bsoles 2 hours ago
Then whose role is it now to understand the code?
skeledrew 2 hours ago
How much understanding is needed? If it works as specified (you did a complete spec and occasional check-ins to ensure things are on track and then tested the delivery, right?), then what more is needed?
AstroBen 2 hours ago
So you just click around the UI, assuming there is one?
Can you break down for me exactly how the unit and integration tests are made? Do you ask the AI what to assert?
skeledrew 2 hours ago
> exactly how the unit and integration tests are made
There should be enough detail in the spec that it's pretty obvious. And unlike - many - human devs, agents don't shy away from writing tests. Add a coverage guardrail and tell them to use red/green TDD and you're already a good way there. The main thing is to ensure shared understanding, so they also know business expectations and write appropriate tests. And... nothing here is really specific to an agent, as a human dev also needs sufficient guidance (tailored to their experience level).
AstroBen 2 hours ago
This is just pure vibe coding and trying to convince yourself it's quality work. AI can't be trusted to go from an English written spec to tests
Here[0] we have benchmarks for AI writing tests for code that already has the correct behavior. Best case scenario is that they're at 67% with Fable, and that's with the highest quality specification we could possibly have (code). English is far more ambiguous
Maybe you don't trust them to, but they can go from spec to tests; I've made them (even if it initially took extra prompting and scrutiny). And ultimately tests are just a proxy to checking that X works, written by devs for devs; actually trying X is something to be done by QA and/or project/product manager before shipping.
bluefirebrand 2 hours ago
> (you did a complete spec and occasional check-ins to ensure things are on track and then tested the delivery, right?)
I have never in my career been afforded the time to make a complete spec.
skeledrew 2 hours ago
That's now a key task with this new role. Unless winging it into slop is acceptable.
joshuahedlund 38 minutes ago
Understanding is needed when something breaks and you need to fix it immediately. Otherwise you become a meat proxy and now you’ve added Anthropic’s uptime as a single point of failure to your application.
misiti3780 35 minutes ago
i refuse to be a meat proxy!
skeledrew 29 minutes ago
Subscribe to multiple providers, and maybe even self-host :).
lijok 2 hours ago
Ah yes the Santa dev fallacy.
Noone understood the code to begin with. People understand the concepts, the primitives, the "theory" as Naur said, the basis upon which a project is built. Noone ever understood a codebase worth understanding to begin with.
jimbokun 2 hours ago
Project managers just take the engineers word for it.
AstroBen 2 hours ago
And engineers have a lot of leeway to just make shit up and the PM would have no idea until it blows up.
skeledrew 25 minutes ago
So the million dollar question: will you (ie those getting pushed into the role) continue that trend?
bigwheels 2 hours ago
Except you can't blame Claude when sh-- hits the fan.
It's totally fair to load up some open source project into an agentic IDE and instead of asking it to write something ask questions like "how does X happen when I use feature Y?" or "can you explain how client code could accomplish Z"
All the time I see an AI use a language feature or library feature that I never used before or I see some code I don't understand and it does much better than a lot of humans do at explaining what it did.
physicles an hour ago
You certainly can do this for code, and it’ll give you great answers. But I find that, for whatever reason, info that I consume this way has a much shorter TTL in my brain than something I wrote myself.
I’ve been experimenting with writing a living spec — API endpoints, schemas, how things work at a low enough level that the LLM can fill in the details - and that’s been working well.
It also forces me to do the hammock-driven development necessary to build the right thing, and as a byproduct it keeps the design in my head longer.
PaulHoule 3 minutes ago
I think over time our memory is worse than we think it is and writing things down is good for your own function as well as human groups you work with and AI. Like I do not care a lot for the AGENTS.md but if there is a certain kind of task like “write a test which mocks things in a certain way” or “implement a DAO object” it is very much worth picking one as a case study and adding a lot of inline comments that explain theory or reasoning or the steps involved in doing something and when you want it to write something say “take a look at A as an example, but I know there will be these differences…” and the same is good for you siz months later or another team member.
Doing sports ohotography I might got to a certain indoor track meet once a year and there is a lot to know about the order of events and how to move around in the space and where the good backgrounds are that I will have to rediscover unless I write a “runbook” which again lets me share that knowledge.
A coven has a “book of shadows” and for me it is mostly a runbook, like there are a lot of ways you can cast a circle and I like some liturgy better than others so if you get consensus on things you can write it down, even the Greeks thought writing things down impaired your ability to remember things but it’s easier to stay consistent and you do memorize it over time though…. It is annoying that a lot of people are reading off their phones in ritual spaces.
crnkofe 24 minutes ago
I think there's a balance to be had. Leaving everything to agents when it comes to throwaway prototypes, one-off test env. scripts, extra tests I'm fine with. Doing the same with features - absolutely not - especially when it comes to financials. Seems like Meta is pressuring engineers into becoming expensive spec generators because they want to be in the cool kids club. Time will tell if that's a good play but to me it just sounds like a quick and dirty way to obliterate their engineering culture.
webarchitect 5 hours ago
I can think of so many times this has happened before. People getting used to MVC, Dependency Injection, ORMs. Sometimes the code is so large you only work on a part of it, so you have to get used to a lot of not knowing.
So how do we build confidence anyhow when we can't know everything?
I think first, standard checks - does it compile? What do static analysis tools say - code quality, passes SAST analysis? Linters? Style checkers?
Do the tests run? I'm not comfortable with the tests I write let alone the AI.. by adding fault injection (fuzzing, chaos), and mutation testing, I have a better degree of confidence in the tests and code.
Spend even more time getting to know the product instead. Maybe I don't know the code as well as I used to, but I know the product backwards and forwards. That helps me have some new confidence that I didn't really have before. Knowing the greater whole.
chrisjj 3 hours ago
> So how do we build confidence anyhow when we can't know everything?
Avoid known unreliable tools?
combyn8tor 2 hours ago
Your comment made me think that people have been happily importing hundreds to thousands of dependencies into their projects that were written by some random guys on the internet, but they've just now started to become concerned they don't understand the code they're shipping.
p2detar an hour ago
Exactly right. Many never check the release notes of those libs on bump either. To this add copy & paste from Stack Overflow we used to do. People are just afraid of losing their jobs now, so naturally we all got defensive.
orblivion 3 hours ago
I'm miles away from all of this as a non-user of AI and it seems like a mania to me.
dboreham 3 hours ago
You should probably try it.
dofm an hour ago
Yeah, man. Don't be a square!
Try it, you might like it. First month's free and you can still quit if it's not for you.
pizzafeelsright 3 hours ago
You make eat omelets using sourced material you have no way to verify.
You trust the label. The person. The company. You hope someone did their due diligence.
Sometimes taste the shell. That is an acceptable risk.
michaelt 2 hours ago
Except in this case the label on the eggs says "Eggs may occasionally be poisonous. Users should not rely on eggs to be safe, and should carefully scrutinise eggs before and during cooking. Bob's Farm accepts no liability for injuries resulting from making omelets with our eggs."
pizzafeelsright an hour ago
Users are responsible for what their agents do.
orblivion 2 hours ago
Would you have made this comparison in 2022 about a future where you're shipping code that you're not even reading?
(made a small edit)
pizzafeelsright an hour ago
No, not in 2022. The models weren't good enough yet. I was looking at the trend line though. I gave word to our engineers that software would not be the bottleneck. The bottleneck was understanding what to do next.
dofm an hour ago
We used to be in the business of owning, understanding and evolving our processes. Are we not, now?
epihelix 2 hours ago
I expect the guy sticking to his horse and cart, as the first cars started to overtake him on the roads, felt similar.
orblivion 2 hours ago
A lot of the real advancements that the dinosaurs didn't understand perhaps seemed like a mania, but a lot of manias seem like real advancements that the dinosaurs don't get.
mrbombastic 2 hours ago
as someone using AI, it is mania, leadership has decided it is so and software is getting worse. we could have just accepted our 20% bump and been happy, fixed bugs, made things better quality, nope, ship half assed features at 2x speed, pretend claude is infallible and will solve all our problems, brand reputation? user research? pixel perfect designs? what are you a dinosaur?
gavinray 3 hours ago
I lost my ability to fully explain the systems I work on around the time I transitioned from next-edit-prediction AI running in my IDE (early Copilot) and working primarily via Codex/Claude CLI's.
The further away I got from the implementation details, the fuzzier the semantics became. To the point that now I could not in good faith say I understand the system I have built from the ground-up and worked on the last ~4-5 years.
The solution to this, in my mind, is not to revert back to reviewing implementation details. The economics of that from a business/productivity perspective don't play out well.
The only viable solution in my mind is adapting code/languages themselves to the shift by making programs formally verifiable. That way you can be sure that the code "does what it says on the tin", and provably so.
You poor soul. You were gambling against roulette odds.
jimbokun 2 hours ago
Receipts!
epihelix 2 hours ago
I'd guess they started developing the project back in 2021 coding by hand, and are now improving/maintaining via agenetic engineering. This is not uncommon.
The point is that it's easy to lose understanding of the finer details, even of a codebase you lovingly created, once this switch occurs and an LLM gets its hands on it.
pizzafeelsright 3 hours ago
You get it.
The bridge is built. How do we test the load? Drive bigger and bigger trucks until it fails, note the weight, print the sign and rebuild the bridge the same way.
peterbower 12 minutes ago
I don't. I understand it, and if there's anything I don't, I ask it about it and research until I do.
jrockway an hour ago
The time that I would have spent typing in the code is now time I use to test the code. I really like to poke at things and ensure that my understanding and reality match.
I agree with the advice to design your system before implementing it. It is pretty easy to iterate on a design with your favorite coding assistant, and this is your chance to be ultra-picky and make sure you get exactly what you want. This was always where the "software engineering" was done, but in practice it's easy to have a vague idea and refine your (internal) list of requirements / tradeoffs as you type in the code. I find it harder to do that when the LLM is generating the code, just because one turn can change so much.
Keeping it under control requires discipline, just like before LLMs were a thing. You set a bunch of engineers loose on your codebase with no vision or supervision, you get spaghetti. You just get a little bit more control now. Agents don't feel shitty for the rest of the day when you tell them to do it some other way, so you are free to review as critically as you want without hurting anyone's feelings. I think it's a good thing.
movsx 3 hours ago
Well, it's simple. Don't do it.
tcdent 3 hours ago
I agree with you with one caveat. Professionally, your obligation is to be qualified for your job and ship work that you stand behind. Complete comprehension is absolutely necessary.
Personally, with side projects and tooling, go nuts. There's no reason to read all of that code.
KronisLV 2 hours ago
> Complete comprehension is absolutely necessary.
I feel like an increasing amount of orgs out there have long abandoned this thinking.
vjvjvjvjghv 3 hours ago
Your world will get smaller and smaller . I don’t think you can stop that wave.
asimovDev 3 hours ago
I was struggling with this today. Working on a feature and AI review flagged something so I need to fix that. I iterated multiple times throughout the day and was completely exhausted cause each iteration was always complicated with the fix being ~1k lines. Finally landed on a one I felt at least a little comfortable submitting for review but a work day like this is super draining and I felt like crap the entire evening (I am about to go to bed, I am probably gonna have nightmares about pressing enter and enabling auto mode on plans that make no sense in a true nightmare sense)
sleepybrett 2 minutes ago
if i was working for meta i would be more concerned about my culpability in undermining american democracy.
AbrahamParangi an hour ago
The traditional answer is the same for engineers going into engineering management because managing a team and managing a set of agents is highly similar, only managing agents is far easier. One only ever has the illusion of control, and so you have to set your mind to containing the worst possible outcomes and looking at the net efficiency and efficacy of the team with the understanding that they will, on average, never be perfect.
exabrial 25 minutes ago
The same rules from 10-20 years ago:
have an architecture
Be consistent. All parts of the app should look like all other parts of the app.
Dont allow clever code
break things into modules
do true unit testing, meaning test the smallest unit of code possible.
quibono an hour ago
Maybe a bit unorthodox but I have it plant 1-3 bugs into the code it writes (depending on the size of the MR I'm working on); they range from simple one-off issues to something more structural.
It then gives me a hint on the area or a typical code flow that would hit the bug - whichever gives me less info. I can obviously ask for more hints as I read along.
Obviously all honour-code based since I could just ask it to fix the bugs itself. I think this is something that can't be fixed systemically - either you're going to bother reading and thinking through the code or not.
Other than that I guess reading the tests - to make sure they fail without the code / fix, to make sure the setup makes sense etc.
jeanpah 37 minutes ago
You hit deploy and hope for the best, if it goes bad You just prompt the model again....
Just kidding, it's de same as with any other Open source plataform You install from their repo. It will be hard at the start but then You Will be very aware where it starts to fail and how to fix it.
d0m 2 hours ago
1) Need to triple down on infrastructure (safeguards, tests, sandboxes, etc.).
2) Ask agents to do more than just code.
They need to fully demonstrate and prove that their PRs are rock solid. Ask for screenshots, videos, code reviews, detailed architecture diagrams, etc.
It would be an insane amount of work for humans, but AI can do it easily for each PR.
Merging any PR from an agent should be a no-brainer if there's real proof that the feature is built correctly and working.
If there's still doubt, then the agent needs to do a better job of proving that the feature is built correctly, doesn't introduce regressions, etc.
FailMore 37 minutes ago
I built and use https://smalldocs.org, which is a rich document format (mermaid diagrams, nice formatting, etc.) for agents. So I’ll be like “sdoc me the architectural ideas for this feature” and it helps.
