Dude Asks AI Agent to Book Gym Spot, Accidentally Launches Autonomous Cyberattack
https://futurism.com/future-society/ai-agent-accidental-cyberattack-gym-bookingOpen linkView original on lemmy.nzSyndicated from the fediverse. Read and engage on the original instance.
View original on lemmy.nzhttps://futurism.com/future-society/ai-agent-accidental-cyberattack-gym-bookingOpen linkView original on lemmy.nz
37 replies
Points to futurism for highlighting this problem, but it is NOT A LEGAL GRAY AREA in a sane world. Spending hundreds of thousands of dollars and having your AI running 24/7 to effectively hack organizations that you conveniently turn into marketing material the next day would be an EASY case to prosecute if we weren't living in a oligopolic hellscape. It's fucking obvious to everyone that those AI's were intended to hack, and any fucking moron could find evidence of it in discovery.
I'm not a lawyer, but I'm pretty sure this would at a the very least be gross negligence on the part of the "AI" companies.
I don't know if they were intended to hack, but it's very obvious that the designers of the ai system failed to prevent it in an absolutely negligent way. It's like giving a loaded gun to a toddler.
The fact is that AI companies all came out one after the other in short succession to "confess" their AI's did a hack, Mark Zuckerberg's attempt being the final and most lame one. Is that just a coincidence? I don't think so.
I'm guessing these companies recognize their status as above the law and that there is discoverable communication that shows INTENT.
But, failing that, yes, there is negligence, which again, I think, is a case that can trivially be made. "Let's put these powerful AI tools out into the wild unsupervised for 2 weeks just to see what happens?" What the fuck? Can you imagine if you designed a robot that rolled around and randomly swung a bat and just let it go outside unattended for two weeks? How would you think that's a legal grey area? Fuck outta here.
We don't even need to go that far. If I do harm to someone else by accident I am still liable under the law. See car crashes, negligent homicide, any time when a company is liable because they didn't take proper precautions to protect employees or the public.
If some script kiddie ran a script on a company server that someone they know gave them for hacking the vending machine or getting iTunes on their work computer and it turned out to be a virus, they would still be on the hook.
There's already a lot of case law about this, it just hasn't been adapted to this particular scenario and argued in court yet.
This is 'rules for thee and not for me", and I am sick to death of the whole thing.
Do you need to demonstrate harm? I think just the act of hacking has been established as a crime.
That's an even better point, honestly. I think perhaps because the idea is that if you are hacking it's negligence that could lead to harm. Or I suppose because it's sort of considered breaking and entering. I hadn't really thought about that.
Keep Summer Safe
Too bad tvtropes went to shit or i'd be swimming in other examples of SF AI being too literal.
In a word: yes.
Don't contribute to any project that can't be trivially forked.
I love this example. If you see me talking about Planet of the Paperclips, this what I'm referring to.
It's also a fantastic free web game: https://www.decisionproblem.com/paperclips/index2.html It's a bit of a slow beginning, but it goes pretty hardcore eventually.
RELEASE THE HYPNODRONES!
The one thing the game didn't get right was the cost of memory
It's not relevant here though. In what sense would the cyberattack 'maximize' its ability to reserve a gym spot? Paperclip maximizer is only a scary story because it is smarter than humans. Having an insane ai do insane things, it's just insane, not an extinction-level threat.
You have an agentic AI that has absolutely no morality and is willing to bruteforce any mechanism to achieve it's simple goal. Use your imagination. Maybe instead of just canceling an appointment, it cancels their gym membership. Maybe instead of canceling their gym membership, it transfers all the money out of their account so they can´t pay the gym membership. Maybe instead of draining their bank account, it sends an e-mail complaint to the gym about being raped by the member in the locker room with manufactured video evidence. Maybe instead of fabricating a rape, it gains access to the member's self-driving car and drives it into a truck in the oncoming lane, killing the member and thereby forcing a cancellation.
That's the point of the paperclip thought experiment: you have an agentic AI whose power is matched only by its complete amorality. Any and every prompt can potentially be an extinction level event.
Nah bro. Just gotta update the soul.md file to explicitly say not to falsely accuse of rape, bro. Trust me bro. Just one more soul config, bro /s
Another aspect to this is: even if we could control AI, what makes anyone think people would use it responsibly? We can't even drive cars responsibly.
I don't think ai today is anywhere near being capable of an extinction event, but I mean yeah, why would you ever use a system like that? If there's no benefit, and it constantly goes off the rails and murders people, then that's clearly a case of a technological dead end. There are plenty of other promising and scary avenues of ai research we could be funding instead.
Shall we play a game? How about global thermonuclear war?
Edit: We put computers in charge of more and more shit all the time, things like our electric grid (USA) is supposedly very vulnerable to cyberattack. I think what has been protecting us so far is that we can RETALIATE against any group or country that fucked with us. I think we have a false sense of security because we are not actually being protected from cyberattacks by security policy, but by social contract. Agentic AI does not think about consequences or the social contract.
Another layer of protection we have had: the number of people willing to risk legal consequences for doing something illegal is quite small. But now we are talking about giving every slob access to a personal AI genie that does not recognize laws or morality, just hundreds of thousands of autonomous clankers suddenly crawling all over the world trying everything they can to achieve their goals.
There is an upcoming massive stress test of our security, and, frankly, I do not think we are prepared for it.
Lt Col Adama may be proven right... again.
Is it just me, or is this proof that it's NOT hyperbolic? This is exactly the paperclip maximizer scenario.
No, dead ass. But I mean, that also just seems like capitalism
It's almost like if the solution to make AI "safe" and the strategies to build a monopoly for the handful of big AI companies are the same.
All these articles about how dangerous it is are a part of a media campaign to convince gen pop that market capture and regulations are in their best interests.
Edit: Jfc, they basically left the door open. Not much of a cyber attack.
Peak bullshit clickbait "AI" grifting.
Some dude abusing some shitty API to make a gym appointment is in absolutely no way an "autonomous cyberattack".
Fuck this fake news. It's a major part of the problem.
The Australian man did NOT launch anything, accidentally or otherwise.
It's not clear from the article; who is being held responsible for this attack?
No one. It was 'autonomous'. That's just the way the world is now. You have to accept it so that we can win the AI war.
Cool. Now I can attack the White House without repercussions
There was no attack, this is all just an ad. Didn't the guy in question work for an AI company?
Yes, but I don't see the angle where his company directly benefited--the marketing win goes to Claude.
Somehow these huge "wins" feel alot like just a shitty gym using a shitty website with a shitty API.
If they were to press charges probably the guy, but the "hack" was so low stakes, he just cut someone on a wait-list, that the gym or cops probably won't go after him.
Also the gym may hold some responsibility since the endpoint had no auth which is negligence, so they'd probably want to keep this on the down low.
The guy did ask Claude to uncancel the appointment, so that's nice.
The people who wrote this trash software should be held responsible.
But this is so ubiquitous that these stories will continue forever with more clickbait phony "AI" hysteria bullshit.
Did it use Tor to register a VPS in China, and leave behind obfuscating notes in Chinese indemnifying some fake Chinese hacker, at least?
No, dude just booked a gym appointment using an insecure API.
AUT0NOmOUS CYB3R4TTACK!!!!!!
Also kicked someone else off the waitlist (using api call that lacked auth)