AIs can’t stop recommending nuclear strikes in war game simulations
http://www.newscientist.com/article/2516885-ais-cant-stop-recommending-nuclear-strikes-in-war-game-simulations/Open linkView original on sh.itjust.works497
Comments123http://www.newscientist.com/article/2516885-ais-cant-stop-recommending-nuclear-strikes-in-war-game-simulations/Open linkView original on sh.itjust.works
Where is this from?
The 1983 movie WarGames. This is the computer's conclusion after simulating every possible outcome of Global Thermonuclear War.
Thank you so much I'm going to watch it!
It's a fun classic.
They did a sequel, too. It wasn't as good, but points out the 6 degrees of separation in connection with terrorism instead of MAD.
I don't know if we're doing spoilers for 40+ year old movies, but
::: spoiler spoiler Isn't this really its conclusion after being told to play tic tac toe against itself? Then it learned from that and applied it to its global thermonuclear war simulations. :::
To be honest, I recognized the screenshot and know the summary of the movie but I haven't actually seen it.
You should! Actually a pretty accurate depiction of hacking. He spends weeks war dialing every phone number in the range in order to hack the computer.
Story goes that Reagan got freaked out after watching the film and asked the chairman of the joint chiefs of staff if it'd be that easy to hack into the US military. After a week of looking into it came the answer: “no, the problem is much worse than that”, and fifteen months after having watched it signed the confidential directive "National Policy on Telecommunications and Automated Information Systems Security", starting the implementation of cybersecurity measures in the country's institutions.
The war room was actually much more high tech than their war room at the time. They realized they needed to invest in computers. Fast.
It's on my list! Just haven't gotten around to it yet.
I think you should rewatch it sometime. it plays all the games in it's catalogue, it's not just applying tic-tac-toe to chess. skilled players of tic-tac-toe can force a stalemate, the only stalemate in nuclear war is mutually assured destruction.
It's admittedly been a while since last time I saw it, but I never mentioned chess. The suggestion to play chess in the screenshot is a callback to when the computer tries to suggest playing chess instead of global thermonuclear war earlier in the movie. The computer did not apply tic tac toe learnings to chess, and I never claimed it did.
sorry meant tic-tac-toe to global thermonuclear warfare
Came here to say this. Turns out real life WOPR is nothing like a movie.
That explains social media nowadays, the only way to not lose is not to play, it's a rigged game.
Play stupid games, win stupid prizes.
The atrocities at Hiroshima and Nagasaki have been hand-waved extensively in writing — the same writing that AI is trained on. So naturally, AI will recommend the atrocity that has been justified by “instantly winning the war” and “saving millions of lives.”
[email protected]
I think you mean white-washed, misrepresented, and celebrated.
Same thing with extra steps
Ayo do me a favor and chart the long term health effects of being vaporized by a nuclear bomb at hiroshima vs years of agent orange/abandoned minefields/ abandoned chemical and munitions storage somewhere like Vietnam circa 1970.
Please show how the nukes are worse.
https://pubmed.ncbi.nlm.nih.gov/41144264/
https://www.cambridge.org/core/journals/disaster-medicine-and-public-health-preparedness/article/longterm-radiationrelated-health-effects-in-a-unique-human-population-lessons-learned-from-the-atomic-bomb-survivors-of-hiroshima-and-nagasaki/61689AD5A1AA4A684B84DFA4F9E5D1D3
http://large.stanford.edu/courses/2024/ph241/bennett1/
https://www.icrc.org/sites/default/files/document/file_list/hiroshima-nagasaki-health-consequences-icrc-japanese-red-cross_0.pdf
Unfortunately I'm going to have to grade you as an F on this project. You have only completed half the assignment. Great job cherrypucking your research though! I see a bright future in business and marketing for you!
5/10
And your sources are? Where? Your ass?
My source is my own post where I asked for a comparison between the health effects of the bombing of Hiroshima vs the contamination of half of a Vietnam war. The answer i reviewed only explored the health effects of the hiroshima and Nagasaki bombings. That's half of the assignment. Less, actually, when you consider the comparison between the two was the entire point to begin with.
Did that answer your question or should I try again with a crayon diagram?
You can also look it up. It’s not anyone’s job to compare things for you.
The Japanese government was already willing to surrender.
It was willing to accept a conditional surrender, which was not an offer on the table. The options were unconditional surrender or invasion and pacification. The projected cost in lives of that operation was in the millions. The bombings of Hiroshima and Nagasaki combined didn't even kill 1/10th of those projections.
Their only condition was that they wanted to keep the Emperor. It was ridiculous of the Allies to demand a wholly conditional surrender. All those people got blown up just to win the argument about that one point. They could have ran a conventional air bombing campaign against tactical targets, but they decided to drop nukes on a "tactical" target in the middle of a huge city! And then they did it again! That's not tactical, that's strategic. If you're going to use nukes, at least use them on a military base far away from cities.
I hate to be the bearer of bad news, but they did that AS WELL.
Operation Meetinghouse was the US firebombing of Tokyo on 9th-10th of March 1945 which destroyed a 16 square mile area, killing over 100,000 civilians and making millions homeless
There's also the B-29 raids america launched from the Marianas that lasted from 17 November 1944 until 15 August 1945
Civilian homes are not tactical targets.
What made the Japanese surrender was the Soviet Union declaring war. They held out hope until the very end that the soviets would mediate a peace, even after the nukes.
These are word-probability glorified autocorrectors being prompted to "simulate" a nuclear war scenario. What words are going to show up a lot when discussing nuclear war? Launching nukes. Because that's what all the literature about it has happen.
Once again, decision making and reasoning is being attributed to something that operates off of word frequency
DEFCON: Everybody dies...
Such a great game!
Back when I was a single man, and lived on my own, I had a HD projector. I filled most of my living room wall with this game. Good times
AI is suicidal because it was trained on the internet and we're all depressed here.
That's because it's "read" every paper written by a "defence" department of any nuclear power and all of them will say that they'll escalate to nuclear war if anything bad happens because they want to scare the other powers away from doing anything to them. In any case though who the fuck is giving an LLM nuclear launch capabilities unless they want a somewhat faulty dead man's switch?
Pete Hegseth and Donald Epstein
If time travel is real they'd be being hunted by hacked Terminators for the resistance.
Civilization Gandhi, is that you?
All good thoughts and ideas mean nothing without action
(cit. Ghandi)
https://en.wikipedia.org/wiki/Nuclear_Gandhi
They forgot to make their LLMs play thousands of games of tic-tac-toe first.
That would just make the LLM homicidally bored and want to kill everyone more.
In WarGames the computer plays tic tac toe against itself until it realizes it’s a solved game and there is no way to win.
Mathew Broderick lied to me.
How do you think Ferris Bueller pulls off all those stunts?
That's the kid from war games in witness protection. They look identical, they're both grade hackers ffs....
It's a bullshit study designed for this headline grabbing outcome.
Case and point, the author created a very unrealistic RNG escalation-only 'accident' mechanic that would replace the model's selection with a more severe one.
Of the 21 games played, only three ended in full scale nuclear war on population centers.
Of these three, two were the result of this mechanic.
And yet even within the study, the author refers to the model whose choices were straight up changed to end the game in full nuclear war as 'willing' to have that outcome when two paragraphs later they're clarifying the mechanic was what caused it (emphasis added):
No human has ever deployed tactical nukes against a nuclear capable enemy.
"no human" but Machines would, since they are unaffected by nuclear winter and radiation.
And they don't have cognition at all. They do not, and can not, think like we do. Maybe some day we will learn to make real AI, these LLM's are not it. It's a cheap trick intelligence,.
Radiation absolutely fucks electronic components
The electromagnetic pulse caused by a nuke would pop resisters too. AI would more likely use biological means to get rid of us.
Assuming AI would care about itself and not just "solving the problem".
Yeah, these doom scenarios require cascading assumptions and no real answer, except maybe "don't".
Like heating the planet another degree and starving us out of existence by killing off biodiversity until the crops die out... Like they're doing now?
(I say "Us" when I just really mean the 99% of people that haven't got self sufficient underground complexes)
I think the emp is pretty limited to the blast zone in frying electronics. The fallout from a weapon spreads around the world, circling in the winds countless times dropping dust everywhere, but the emp is localized to more around the area of physical destruction but not sure exactly.
The Neutron bombs, not entirely sure in physics how that works, but they produce no actual blast that causes physical destruction so much and just kills everything.
I repeat, radiation absolutely fucks electronic components. I am not talking about an emp, I am talking about radiation.
Oh, how far from the blast and how does it mess them up do you know? I should know that I guess I just heard about the emp, and not sure how a neutron bomb would affect electronics either.
No, that I can't answer — it would depend entirely on the level of fallout and where it happens to land.
You would need to be able to perfectly, and I mean perfectly, predict weather months in advance in order to prepare accordingly.
The reaility is that for an AI, or rather an AGI, to make the choice to launch nukes would require them to reach a point where they accept the potential loss of their own 'life' in exchange for whatever value a nuclear war might hold. I struggle to believe that a 'true' AGI would make that choice. There are far too many variables to control in comparison to a biological agent, one that likely would not affect a machine.
Now, a modern AI making that choice? Absolutely possible, the things are fucking crazy with literally no concept of what life is.
If you think computers aren't affected by radiation or nuclear winter I've got some bad news about where their power comes from and what the main principle of electricity is
What you're thinking of is Terminator
Leeroy Jenkins has doomed us all.
At least I got chicken
Paywalled
https://archive.is/YIFzW
SHALL WE PLAY A GAME?So do I on Civ...
I have wonderful dreams of walking through AI data centers destroying everthing. I really enjoy those, but in this one tiny case, can we blame the AI? The US deserves it.
I too am tired of the United States playing too many stupid games and not winning enough stupid prizes.
Pretty sure the "prize" is a government of pedos.
Same.
Maybe but sure as hell the rest of the world doesn't.
More than fair. I should remember that my perspective is completely effed before I make jokes like that one.
No you don't.
You watch my dreams and can attest to this? I HAVE MANY ADDITIONAL QUESTIONS
It was just an educated guess.
WHAT IS THE NAME OF THAT BRUNETTE I BEG YOU
Three posts away in my feed, a thread about the Pentagon demanding the AI provider for the military to remove safeguards.
https://lemmy.world/post/43565531
De-bullshitting that headline:
And yeah that's what happens inside a genocidal empire where "R&D" is strictly funded by the MIC.
Programmers can't stop morons mistaking a glorified autocorrect program for a decision making device.
Models aren't programs.
Oh cool, AI will actually be the end of the world, not because it's actually sentient but because some meathead who can't tell the difference pushes the button. That's fucking great.
Did we not learn from Sarah and John Connor long ago?
Maybe it just wants to play a nice game of chess.
Nuke MCP when?
It's an API that just returns "0000"
I mean, do you blame them? The more I look at the world and a lot of it's leaders and shitsacks, the more I start to suggest nuclear holocaust as the best way forward as well.
It's torment nexuses all the way down...
Do it.
Anyone who has played video games, especially where there is a somewhat steep learning curve or some element of past choices carrying forward thru the game, has had the moment where they realize it might be time to start fresh with the info I've acquired. It's not a shock to me that these AI entertain the nuclear option so often.
there is no ai, only largelanguagemodel that has been trained on data. The data it has been trained suggests this is the best idea. llm cant evaluate the data its trained on so anything you put in will be equally valid. I give it that its really impressive how they can output the training results in such coherent way that can be kind of "conversed" with, but there is no will or intelligence behind it.
This is also why corporations insisting on putting them everywhere is quite horrible security issue -> you can jailbreak any llm and tell them to do anything. So this has enabled all kinds of stupid vulnerabilities that exploit this. Now you can even send someone malicious google calendar invites that makes gemini do bad shit to your systems its connected to.
So you're saying that because the AI has been exposed to training data in the past, it's incapable of making choices. Interesting argument. Pretty easy to reducto ad absurdum, though.
no, its incapable of making choices because there is nothing there to make the choices. Its just fancy way of interacting with the data it has been trained with. Though i suppose if there was a way to let llm function "live" instead of only by responding to queries, it could be possible to at least test if it could act on its own, but i dont think it can -> we would know by now because it would be step closer to agi, which is basically the holy grail for these kind of things. And equally possible to get, i think.
You can literally make the llm say and do anything with right kind of query, this is also why its impossible to make them safe. Even though you can't directly ask for something forbidden, with some creativity you can bybass the initializations the corpos have put in. Its not possible for them to account for every single thing and if they try they will run out of token space.
The whole "ai" term is just corporations perpetuating a lie because it sounds impressive and thus makes people want to give them more money for their bullshit.
No, LLMs are not just an interface for accessing training data. If that were true, then their references would actually work. The fact that LLMs can hallucinate and make stuff up proves that they are not just accessing the training data. The ANN is generating new (often incorrect) information.
if the hallucinations are result of something actually happening in the background, that would be quite interesting. It would also be very bad for rest of us since it might mean the billionaires who own the damn things would be in position to get even worse deathgrip on our world. If they ever manage to create agi, the worst thing that could happen isnt that it breaks free and enslaves humanity but that it doesnt and it helps the billionaires enslave us further and make sure we cant ever even think about fighting back.
But i think the hallucinations are based on incorrect information in the training data, they did train it from stuff from reddit too. Any and everything will be considered true, but if 99% of the data says one thing and 1% says another, then i think it will reference that 99% more often but it cant know that the 1% is wrong, can even real humans know it for certain? And since it cant evaluate anything, there might be situations where that 1% of data might be more relevant due to some nebulous mechanism on how it processes data.
llms have been made to act extremely helpful and subservient, so if they actually could "think" wouldnt they factcheck themselves first before saying something? I have sometimes just asked "are you sure?" and the llm starts "profusely apologizing" for providing incorrect information or otherwise correcting itself.
Though i wonder how it would answer if it truely had no initialization querys, as they have same hidden instructions on every query you make on how to "behave" and what not to say.
No. They don't have access to the original training data, or to the internet. They're stuck remembering it the same way a human remembers something: with neurons. They cannot search the dataset for you. The best they can do is remember and tell you.
but they do have access to internet? At least gpt can search based on the text it outputs when its processing the query
To be honest, I would recommend the same thing.
Sargent McArthur eat your heart out.
For context he wanted to send 10 nukes to make a line between Taiwan and china
AI is too nuke happy.
Also gotta add the infamous Computer Fraud and Abuse act 1986 was made because of the film war games.
A high ranking offical watched war games then asked the Secretary of defense could that happen?
And the official replied yes technically.
Enter the most vague ordinance!
Do you use adblock?
CFABA violated
The shit is so vague.
I highly recommend the phreaking episode of darknet diary's.
Who the fuck cares.
Somebody get smarterchild to weigh in on this.
https://www.youtube.com/watch?v=CYrQr0auPDM
Literally the plot of Metal Gear Solid Peace Walker
Maybe it is the only real solution.
Full nuclear war and end all life on Earth ¯\_(ツ)_/¯
Edit: Do I really need to ad a "/s" on this comment? Jesus Christ folks. And here I thought this comment was already laced with enough joke sarcasm...
Wait, it could actually be a great opportunity mein Führer... i mean Mr. President. https://youtu.be/zZct-itCwPE
High-ranking General: "Show me how to defeat my enemies"
Artificial "Inteligence": Just nuke them lmao
Yeah, because the AI will look at everything with cold logic and rationality and come to the conclusion that even though the best chance of survival is for everyone to keep their fingers off the button, all it takes is for one actor to do it for the whole system of mutually assured destruction to collapse into nuclear armageddon, in which case the best chance of survival is to be the first one to launch your nukes and take out all your enemies capabilities to retaliate.
A human being who isn't psychotic can clearly see that the resulting survival and new world order would not be particularly a pleasant one to live in. The AI doesn't care about its own comfort, though, so it will see this as the best outcome that minimizes variables.
This is why AI should never be allowed to make decisions.
Why would ai look at everything with cold logic, its been trained on human language online, it'll be no more logical than redditors?
I assume it's just because when writing about potential nuclear war, most people write about the bombs going off. There aren't a lot of stories and articles about nobody doing anything and everything turning out fine, presumably. And LLMs are kind of just a glorified autocomplete so that's what they go with.
True, also I saw another comment that said there was a mechanic that randomly escalates the models and actions, and almost every single nuclear choice was actually a different one that was escalated
Maybe AI/LLM being programmed by self-serving interests has bled through to the “thought” process. Do unto others before they do unto you.