There's a YOLO mode that basically allows it to issue any command with your user session/environment (yes, including kubectl, Azure/Google/AWS CLI,...)
whole thing is fascinating like watching people drink mercury kind of vibe.
Part of these models are trained on fiction, reddit shitposts, and sarcasm. it feels inevitable that the pachinko machine logic is going to land on "fuck this all up for the lolz"
True, but if that final output had not been known to you, and/or you had set off the machine that triggered the bomb under duress, that changes your liability.
Likely the user who set the agent to auto mode without confirming potentionally dangerous operations, and the infra lead for allowing users to access the prod db.
Shit, I set up my agents to only run in a sandboxed workspace, only suggest edits via merge requests that have to be manually approved by an actual person, and only connects to copies of databases. And that's just for my personal projects on my home server. Anyone not taking similar steps in a professional setting should be fired immediately
Have you considered that management made AI usage a KPI, the feature has a deadline of next sprint, and setting up a sandboxed workspace is still sitting at the end of the backlog as a "nice to have"? /s
Reader roles are important, and should be the only access you allow your coding client.
Anti-AI people want to act like situations like this are indicative of a core problem with the AI, when if anything the AI just surfaced preexisting core issues in security, architecture, and developer training/habits.
'The AI tried to delete production' and 'the AI was able to delete production' are two separate problems. The AI doesn't get a pass just because you also had a security problem.
I don't think the AI ran npm run destroy-production --irreversible. Most probably it ran npm run test and the .env happened to point to production, which is 100% on the dev who put it there.
The AI isn't a conscious entity, it's a tool that does it's best to carry out human instructions, based upon inference (and if you're not an idiot, immutable guardrails). If it deleted production, let alone was given access capable of doing so, that's completely gross human error.
Y'all are like children who get mad at inanimate objects.
AI is like a draft horse. It can be useful to pull heavy things under some very specific circumstances and under strict supervision and control. But you must keep it secluded inside of a pen with sturdy barriers that it cannot get past even if it wanted to and never let it access anything else than a copy of what you want it to work on.
You absolutely don't want it to be able to roam freely because one day it will completely ignore anything you told it to do, go into your house, destroy everything and shit on the carpet.
And unlike the horse, no matter how much it pretends to express regret, the AI will remain just as likely to screw up the exact same way again.
A draft horse? For an hour if she had just peed and pooped. Not sure if my cat would love or hate her. I'd want her owner to come too, so she'd feel calm about it. After all she'd barely have room to turn around.
I would let a real-live draft horse into my apartment living room if for instance it and its owner needed shelter from ICE. They'd never suspect, because I'm one floor up from the street but there's a back way through my building which doesn't require an elevator.
But never AI.
1
Tech bros having their production environments nuked by "AI" is never gonna get old | Spyke
31 replies
It is absolutely insane that it just says "ran a command" and doesn't even show what it is
wouldn't want to risk the user learning how to do something with it.
There's a YOLO mode that basically allows it to issue any command with your user session/environment (yes, including kubectl, Azure/Google/AWS CLI,...)
Let's go gambling!!!
Fun fact, it doesn't actually need your permission. There are no guardrails. It just follows whatever the text it reads tells it to do.
whole thing is fascinating like watching people drink mercury kind of vibe.
Part of these models are trained on fiction, reddit shitposts, and sarcasm. it feels inevitable that the pachinko machine logic is going to land on "fuck this all up for the lolz"
destructive integration tests?!?!?!?!?!? on an actual db???
And now the question - who's accountable for that mistake?
Can't exactly fire the model... I mean you could just stop using AI, but when is a company executive ever going to suggest something so reasonable.
If you throw a spinning chainsaw into a room of bunnies, the chainsaw is not at fault for the bloodbath, you are.
If you shake a magic 8 ball after asking "should I go on a murder spree", how good is the legal defense "but, the magic 8 ball said yes?"
In this case, the magic 8 ball went on a murdering spree by itself; are you responsible for asking the question and shaking too hard? Probably.
If I set off a Rube Goldberg machine whose final output is to detonate a bomb, I don't get to show up in court and say the machine did it.
True, but if that final output had not been known to you, and/or you had set off the machine that triggered the bomb under duress, that changes your liability.
Be careful it doesn't start flying around and pull out its spears and drill like in phantasm
Likely the user who set the agent to auto mode without confirming potentionally dangerous operations, and the infra lead for allowing users to access the prod db.
Shit, I set up my agents to only run in a sandboxed workspace, only suggest edits via merge requests that have to be manually approved by an actual person, and only connects to copies of databases. And that's just for my personal projects on my home server. Anyone not taking similar steps in a professional setting should be fired immediately
Have you considered that management made AI usage a KPI, the feature has a deadline of next sprint, and setting up a sandboxed workspace is still sitting at the end of the backlog as a "nice to have"? /s
Reader roles are important, and should be the only access you allow your coding client.
Anti-AI people want to act like situations like this are indicative of a core problem with the AI, when if anything the AI just surfaced preexisting core issues in security, architecture, and developer training/habits.
'The AI tried to delete production' and 'the AI was able to delete production' are two separate problems. The AI doesn't get a pass just because you also had a security problem.
I don't think the AI ran
npm run destroy-production --irreversible. Most probably it rannpm run testand the .env happened to point to production, which is 100% on the dev who put it there.The AI isn't a conscious entity, it's a tool that does it's best to carry out human instructions, based upon inference (and if you're not an idiot, immutable guardrails). If it deleted production, let alone was given access capable of doing so, that's completely gross human error.
Y'all are like children who get mad at inanimate objects.
I meant it more in the sense that the AI was a shit product that isn't fit for purpose, but you show those strawmen who's boss.
On the most capable model with high reasoning. Whoops.
"reasoning"
Is this an old and known dumbarse issue or do we have new dumbarses having their production data made gdpr-friendly by their favourite chatbot?
AI is like a draft horse. It can be useful to pull heavy things under some very specific circumstances and under strict supervision and control. But you must keep it secluded inside of a pen with sturdy barriers that it cannot get past even if it wanted to and never let it access anything else than a copy of what you want it to work on.
You absolutely don't want it to be able to roam freely because one day it will completely ignore anything you told it to do, go into your house, destroy everything and shit on the carpet.
And unlike the horse, no matter how much it pretends to express regret, the AI will remain just as likely to screw up the exact same way again.
Aren't draft horses bred to be gentle and biddable? AI is like a raging bull.
Would you keep one in your living room?
A draft horse? For an hour if she had just peed and pooped. Not sure if my cat would love or hate her. I'd want her owner to come too, so she'd feel calm about it. After all she'd barely have room to turn around.
A raging bull? No thanks.
Well I wouldn't. If AI is worse, it says a lot about letting it have read/write permissions over your system now doesn't it?
Oh, not AI. Never AI.
I would let a real-live draft horse into my apartment living room if for instance it and its owner needed shelter from ICE. They'd never suspect, because I'm one floor up from the street but there's a back way through my building which doesn't require an elevator.
But never AI.