Pentagon says overreliance on AI contributed to missile strike on Iran school (bloomberg.com)

But don't worry, the other P AI system will ensure that the first one doesn't do mistakes...

> Some US intelligence analysts took notice, the officials said. Bloomberg reported in June that one analyst detected changes to the site as early as 2019, and logged remarks about the modifications in a system that wasn’t connected to the primary military intelligence database, which informs targeting

-------------------

Maven is an untested tool for rapidly selecting targets under time pressure during a war. It's reason for existing is speed. If you're in a shooting war, the ability to select good military targets faster than your enemy, but not necessarily with zero error, is an advantage.

Planning a first strike is a different situation. Time pressure should not be a significant factor. Humans should have been doing their due diligence on every single target. If an untested tool for rapid mid-battle target selection was relied upon completely because of a hasty executive order, then the responsibility falls entirely on the executive.

Except laundering our biases & discriminations, the unaccountability seems to be the purpose of a huge huge swarth of AI usage. To eliminate accountability. That sure seems like A Feature Not A Bug to the worlds worst people.

> The Pentagon chief last year slashed offices that didn’t contribute to his goal of “lethality,” including the group that assists in limiting risk to civilians, known as the Civilian Protection Center of Excellence. Around 200 employees who worked on the issue, including at that office, have been reduced by about 90 percent, according to two current and former officials and a person familiar with the effort. The team that handles civilian casualties at Central Command, which oversees the Middle East, has dropped from 10 to one.

Negligent homicide in pursuit of their DEI goals, basically.

(setting aside when civilians are purposefully harmed which is a conversation about morals rather than the capability to make correct decisions)

What I am getting at is, has AI really made the problem worse or has the mechanism just changed?

This was a totally expected outcome, similar to how CV cameras arrest the wrong minorities in neighboring states 100s of miles away because it told the officer it was them; or how health insurance companies use ML algorithms to deny Americans care.

It's going to happen in other sectors of life and it will continue killing innocent civilians.

Really good time to do various crime.

> That didn't happen.

> And if it did, it wasn't that bad.

> And if it was, it was AI's fault, not mine.

It's gonna say something like "they were all military-aged males". https://en.wikipedia.org/wiki/Military-age_male

https://www.propublica.org/article/hegseth-civilian-harm-mit...

Mind you Israel is currently funding a lobbyist organization in the UK to flag Médecins Sans Frontières (Doctors Without Borders) as a terrorist group (because many years ago a couple of people who had affiliations with other anti-Israel groups were found to be also affiliated with MSF).

My brother in Cthulu, they're the ones making the life or death decision that killed people, who else should be to blame?

>But more relevant to HN, this couldn't have happened without technology professionals working for Palantir.

If you wanna go the historical tech-tree scapegoat route then none of this would have happened if DARPA wouldn't have invented the internet, or if Thomas Edison wouldn't have invented the lightbulb, or if the fish haven't ventured onto land and evolve into us land mammal apes who invented sharpened sticks as murder objects.

Would you rather not have someone make good software that does not misidentify schools at sites? Or should we outsource this to worse engineers who don't care about morals and don't find this metric meaningful?

So yeah, the military and whoever decided that a war was a good idea are to blame.

People working at Palantir are as much responsible for the killing of these 123 children than seatbelt manufacturers are responsible for the people who died because their product failed. That is, because they didn't do their job correctly, people died, definitely worth feeling bad about. However, should people stop working in seatbelt manufacturing because of it? Absolutely not, we need seatbelts, because without them, things would be worse.

Palantir, or whoever does target identification, is like the seatbelt of the reckless drivers of the militaries, who, for the lack of precise information, would just carpet bomb the place. It is not a hypothetical, it is how the US has worked before they had precision ammunition.

This is essentially a trolley problem. The war is the trolley, people who work for Palantir are the ones pulling levers. And if there is no one to pull the lever, the trolley is gong on a track with a lot of people on it.

> I encourage others who work in the public sector, whether directly or through a contractor, to read and internalize Don't Get Distracted by Caleb Hearth.

I know a guy who works for well-known hardware company. He isn't a software dev, but close enough, works with Verilog on hardware design and verification. One day he tells me how he is working on video processing module and he doesn't know if that module will be part of surveillance system, missile guidance or nanny cam. It's possible that even middle management doesn't know more than him, and very likely the module will find uses in multiple products. He is a small cogs in a large machine, maybe his work will save a baby's life, maybe it will kill a person, he cannot know.

https://en.wikipedia.org/wiki/My_Lai_massacre#Court_martial