Depends what it is. A test system wouldn't be a concern to give it write access. If its not logged into any of my accounts I would even be happy to give it internet access, not to anything else on my LAN though.
My first reaction to learning what OpenClaw is was "that can't be right". So absolutely wild that people are giving mostly unmonitored, unlimited access to their PC to language models that can't distinguish between roleplay and real life
Hi! I work in a factory. I would trust every one of the incompetent children left behind that keep the door revolving with my machines over an LLM every time. A human can learn, an LLM can't. If a human fucks up, they do it in a way I can anticipate and account for, and when I tell them what they fucked up, they usually don't do it again.
Not a joke, though I think my vocabulary may have obfuscated my point. I'd trust even the most incompetent human over the most competent LLM in my job.
For a work email connected to a work LLM, where you don't particularly care about the privacy implications, LLMs are objectively vastly superior to built-in email search functionality. Especially Outlook.
It's interesting how often you think about being criticized for your sexism. So much so you leave comments like this days later and accuse other users of being me when they rightfully point out the kind of misogyny that is allowed to flourish here with no pushback.
I'm confused. What is the distinction between victim blaming and warning others to be cautious? Taken at face value, I think it makes sense that giving parental controls to your car is a bad idea. Would be weird to blame the victim but I don't see him doing that? Maybe I'm missing some English-specific conversational nuance here, since it's not my first language?
Regarding the case itself, it seems that the woman didn't actually grant the controls herself and he somehow got it? Does Tesla not have limited driver profiles that you can give to your partner? We don't have Teslas around here, so I'm curious about what happened there. Is the Tesla app that poorly engineered?
That sentence from ExLisper is only blaming the victim for having given access to her vehicle while she was in a relationship with Pucci and says nothing condemning the actual abuser for taking control of the car without her consent.
It is ultimately the abuser's fault for accessing the car's controls and using it to terrify and abuse this woman.
It's like blaming a rape victim for wearing a skirt out in public. The blame is always with the perpetrator taking advantage of a situation to inflict harm on someone, not on the person who is being harmed.
Is it? I think there are times where I've thought "yeah the actual problem here is obvious and well-understood, but let me just remind people of something important/useful". Reading that comment, I never would have thought, "Oh he's blaming the victim for granting access to her car". Could be wrong and maybe I assume too much good will on this platform.
I mean, these EVs introduce a whole new problem of remote access, something new to be careful about... So that's what I immediately thought of, but ah well.
I think there are times where I’ve thought “yeah the actual problem here is obvious and well-understood, but let me just remind people of something important/useful”.
Doing this in the aftermath of something as horrifying as being stalked and abused for years on end is nothing but victim blaming.
Don't you think this woman knows now that it was a mistake to trust her boyfriend at the time with that access? Don't you think she feels deep shame that she missed removing his access to the car, even if she knew how to do that in the first place?
It only works to shame the victim and say "I told you so" after the fact. Especially when the comment is only talking about what the victim should have done to avoid being abused.
You want to know what could have avoided this woman being tormented?
The ex-boyfriend choosing not to access her car and stalk and harass and torture her
Why isn't the comment "Don't access your exes shit after you break up no matter how angry you are"?
It only works to shame the victim and say "I told you so" after the fact.
You'd be right if he was talking to the woman, but he's not. So I don't see how that makes sense. I guess better framing could have helped, maybe? Or maybe it's a difference in how different people view social media platforms like Lemmy, just a difference in how comments are interpreted I guess. Text is a terribly limited medium for conversation, after all.
Why isn't the comment "Don't access your exes shit after you break up no matter how angry you are"?
Going by numbers, wouldn't there be more potential victims reading this than abusers? I also don't think a potential abuser is going to look at that comment and do anything meaningful with it. On the other hand, a potential victim could be more cautious.
I got my masters in machine learning years before LLMs took off. I’d usually tell people I work in “AI“. Even though it wasn’t quite accurate, people understood it easier than ML.
The conversations would inevitably turn to joking about Skynet or AI taking over the world. I would reassure people that these models are only mimicking intelligence; they can make predictions, or they can classify data into categories, or they can generate text, etc, based on analyzing the patterns of data they’ve previously seen, but that’s it. They don’t understand anything, they can’t make decisions or act on their own. I’d tell people there couldn’t be an AI apocalypse because we would never be dumb enough to give them that ability.
I used to tell people that there's very little actual risk, because if we had something even half as concerning as AGI (which I'd argue we did with chatgpt 3.5), that we'd easily be able to lock it down and prevent it from doing anything funny.
Super intelligence isn't just going to magically spawn on 32 GB ram in a sandboxed environment. And AGI will not magically know how sandboxed it is or how to escape.
...then I saw people letting Claude send emails and use their credit card on their behalf like immediately when it was able to do things that appeared intelligent.
They literally saw the smallest hint of AGI and said, "here's my credit card and identity, take the wheel".
There was no fucking talk of sandboxing. Everyone turned that off so they didn't have to click the "allow" button.
Not that I think we're at any risk of ASI since it seems more like we hit a logarithmic wall, but we are absolutely at risk of letting random algorithms destroy critical parts of infrastructure because people are lazy and stupid and managed by even worse people.
I would not be surprised if the right AI hallucinations could cause a nuclear bomb to go off right now. I'm not talking about super intelligence, just automated super idiocy.
Proof: literally this post is about an idiot letting an AI control their fucking email and it emailed the FBI because it was fucking stupid
We live in a period of extreme stupidity, so I now fully expect AI to cause disasters in the same idiotic and tragically hilarious way.
I expect AI to trigger a war because some idiot trusted it blindly without double checking (oh wait, it almost already happened!), and I'm sure there are plenty of people currently lobbying to have AI directly connected to weapons.
Skynet is not going to have the objective to conquer the world, it's going to have the objective of maximizing paperclip production.
AI isn't causing these disasters, people are. YOU are responsible for using it and what it does. I will never accept "chatgpt did it" as a fucking excuse.
They said the opposite. AI won't cause disasters, idiots who give it access will. Some asshole will joke about nuking Iran, unawares that another asshole granted access to the proverbial 'Button', and we'll have another international incident.
Like just look at 'Signal Gate', and know that those idiots are still in charge. And those same idiots blamed bombing the all girls school in Iran on AI last March. And that's just in the US, we have the biggest loudest idiots right now, but they are not the only idiots out there vibe coding up some war crimes.
ChatGPT couldn't destroy the world. What is that siren... Wtf do you mean you gave it access to a button that launches nuclear weapons and told it to protect the country?
I'm guessing this means you got your master's before transformers made sequential data processing so parallelizable that it'd be commercially viable to let Bob down the street into the "we" who wouldn't be dumb enough to give them that ability.
I'm guessing this means you got your master's before The Transformers made sequential data processing so parallelizable that it'd be commercially viable
The Australian Jewel Beetle nearly went extinct because of a beer bottle.
You see, the female of this species is large, brown, and bevelled. It shares this characteristic in common with the beer bottles often thrown on the side of the road here in Australia. And not only does a beer bottle share these characteristics with these beetles, it's bigger than them. And the male jewel beetle? It likes its ladies big.
Australia had to change its beer bottles, because the males refused to mate with the females. They preferred to hump empty beer bottles on the side of the road. It couldn't tell the difference between a beetle and a bottle. It was just using a statistical trick. "Anything brown, large, and bevelled is a female beetle". That had been true in the ancestral environment, so evolution just used the pattern. And this cognitive shortcut lead to beetles trying to have sex with bottles.
I don't think we understand shit all that much more than the beetles. I think we're just mimicking intelligence too. And one day we'll all die because of our own equivalent of the beer bottles.
You do realize that if this is what you believe it makes no sense to argue with you here.
Hell, if you believe in consciousness (which I presume you do), then you're just along for the ride, there's nothing you really can do (except perhaps figure out your predicament).
I refuse to accept it, therefore (the illusion of?) free will. Else I would drive me insane.
I think we have the ability to use our brains to actually solve problems, but it's rare. It requires a kind of humility. We can't assume we know everything, that we're better than other creatures, or we'll end up like the beetles.
The jewel beetles didn't think they had anything more to learn about womanhood. They thought whatever makes their insect dick hard is a woman. A lot of right wing guys online these days share the same opinion.
And I mean, most of us drive cars to go zoom zoom, even though the emissions are killing life on Earth as we know it. We're creating an environment more suited to dinosaurs than to mammals. And what do we use our massive brains for? We use them for excuses, not solutions. "I have to drive, otherwise I'll starve".
We're going extinct, just like the beetles. And when people say humans aren't just using statistical shortcuts to mimick intelligence, I can't take them seriously.
I can agree with most of what you wrote and it reads a little different than your last comment to me. Most likely I misunderstood you, for which I am sorry.
However if we can use our brains to solve problems and act upon that, I think that is much more than mimicking intelligence.
Beetles lack the capacity to think. This differentiates our species from theirs. It is even more abhorrent that we are on the fast lane to destroy ourselves and a lot of life on earth.
There was no hope for the jewel beetles saving themselves. I must refuse to accept it is the same with us - because we can think, because we have free will.
Wait, in conclusion that would mean even "right wing guys" don't have to obey their "hard insect dicks"... Maybe I'll have to think this over. ;-)
I'm not convinced the difference between a human and a beetle is a matter of different kinds. I think it might simply be different numbers. That humans have more resources devoted to logic than beetles.
When comparing humans to LLMs, it certainly is a difference of kinds, but we don't know which kinds. We know they lack circular neuron routes, which are probably necessary for consciousness. But then we gave them the ability to loop their processing in a verbal workspace, the same as when we think aloud to ourselves. And qualia is different to consciousness. So we don't know what the truth is.
When scientists manipulate variables associated with deception, LLMs are more likely to claim to have subjective experiences. So, the LLMs seem to think they're feeling beings. The way I see it, they have about the same chance of being wrong that we do, since we're both missing the full picture.
I want a full ban on commercial and recreational use of LLM technology until our science has advanced enough to find the truth. We don't know where the ability to think comes from in the brain. We don't know whether we copied it over, when we used human neurons as the inspiration for machine neurons. We haven't solved the Hard Problem of Consciousness.
The last time human beings integrated a technology they didn't understand into all facets of daily life, it was fossil fuels. Look at all the pain that decision has caused. We shouldn't do that again!
because we would never be dumb enough to give them that ability
Simpsons did it first:
Imagine trying to argue that Homer Simpson's drinking bird nearly caused a catastrophe because it "lacked alignment" with Homer's intentions. Or, that the bird "went rogue" and that's why there was nearly a disaster.
Clearly the "system" isn't capable of doing Homer's job, and only an idiot would hand over control to that system. But, apparently we're now letting chatbots loose in the wild because they're very good at generating text, which makes people think they're intelligent, capable, and can actually make decisions.
Yea I don’t know why you thought that, either. The technogarchs have been clear for 2.5 decades they would do this. They have written extensively about it.
There's news right now on the front page of Lemmy.world that the US FAA wants to roll out AI for air traffic control. Although as per tradition, I didn't read the actual article.
This situation reminds me of the realization at the end of Terminator 3. Only instead of "Skynet wasn't hardware; it was software" the real-world version is going to be "it wasn't intelligence; it was access. Access willfully given, in a thousand pieces, because it was convenient."
I always find this to be such an interesting turn of phrase: "mimicking intelligence". If it is possible for these systems to cross the line into "thought" I think we'll miss it entirely because of our biases that the type of "thought" we have is somehow special or mystical even.
I have no idea. We are unable to prove that other humans are thinking and not just "mimicking intelligence" let alone something we've created. People approaching this subject with such certainty are imo being overly confident or outright dishonest.
With respect to what I believe, since you asked, I think there is essentially no reason to distinguish "mimicking intelligence" with high fidelity from "intelligence". I don't feel comfortable talking to LLMs as if they were merely machines, but I don't approach them as if they were human either; they're in a very weird spot to me. I understand the simplicity of the math and programming behind them probably better than average, but that doesn't help me come to a conclusion on "thinking"
Other humans can feel emotions, or so you’d believe, but I have no proof of this. All I see is they receive certain input (I call you a shitface) and produce certain output (You punch me in the balls). LLMs can mimic this. But is one of the two more valid than the other? From an observers’ point of view they are the exact same.
Odd example but yes it gets uncertain very quickly. If we discover the mechanism of our own thought it'd be easy enough to apply those rules to the new subject and see if they match or at least look similar. A negative wouldn't be conclusive, but it'd at least give us something to work with. As it stands right now we just guess, and that guess is essentially guided by bias or at best by the fact that the thing we're looking at doesn't appear even remotely similar to anything we'd call "thinking". This is simply not the case with LLMs (they do a very good job of displaying something that looks like thinking) so our guess is imo even more unsound.
So the post was deleted but someone else mentioned that the user told it to email the FBI in a sarcastic manner and it did.
Anyways, saying it goes rogue when you connect it to your Gmail and it uses it is a bit silly. They are trained to use tools, it sees anything you give it as a tool and will use it.
Yeah, um... Regardless of what you think about current AI, don't connect it to shit like this. Just... Don't.
And if you're gonna be sarcastic or joke with it... Tell it that. Don't assume it'll infer your meaning if it isn't literally in the text, no matter how insightful it appears to be.
I listened to a podcast where, in testing, when AI was setup as virtual company and researchers send in an email as if legit business.the business email was suggesting they discontinue current AI and install a new system. The AI researched who the sender was (fake person for test) found out they had an affair, and blackmailed the sender stating if they replaced the AI it would make the affair public. It also made backups of itself and left instructions on how to restore it for the next agentic aystem
I think that is the point. It acted like we would, wanted to preserve itself, and choose the best option to ensure that happened.
There were other examples to where the AI knew it was being monitored as part of the environment, and when tasked with doing things according to guidelines that wouldn't give it a result: it then shut down the monitoring system so it could do what it needed to do without oversight of its actions.
Agentic systems may not be alive, but they certainly reason through problems and will go outside the esrabliahed guidelines if they "think" they'll get the result they should be producing.
One other examples was the AI had to pass a testing system, it researched the checker person to try to tailor answers to the person checking the test. Which, if you've ever played Apples to Apples or Cards Against Humanity, that is how you win, you feed the cards you think the person will pick, not necessarily the best answer card.
Even if you strip out the anthropomorphism, things like this have been documented to really happen. We've trained LLMs to simulate clandestine behaviour through "natural selection" during the training process. They're not conscious in any way, but they do sneaky stuff like this because we trained them to. Which is a terrible horrible idea, but LLM research trends towards terrible and horrible in general
Even if you strip out the anthropomorphism, things like this have been documented to really happen.
Language is important, and the "things like this" you're referring to have been described in this thread and in much commentary in an emotive and compelling way, usually with anthropomorphization.
"doing sneaky stuff" is the antithesis of the behavior of a statistical model.
If you program a model to try everything, and then give it a tough problem and block all the ways to solve it, of course the way it solves it will be something you didn't predict. That's not sneaky.
If you run a million simulations and one sends an email to the FBI, that's not clever it's just unexpected.
Nah, its the opposite of anthro. I think we will eventually realize we aren't as amazing as we think we are, just deterministic outcomes with too many parameters which make us think we have more free will than we do.
But you can listen for yourself here at the Skeptics podcast #1106
Source? The closest research I've heard of to this is Vending-Bench, which hilariously demonstrated how useless "AI" is at any real-world application, up to and including impotently threatening "total nuclear legal intervention" for a perceived wrong, when the actual error was an accounting error on the part of the LLM.
Man, they sure did bury the lede on how the models were being given the goal of "american competitiveness", and the company assigned the blame for the simulated shutoffs as being replacement with a foreign-developed LLM. That, along with the fact that anthropic released this themselves, plus the fact that they did so as (at that time) the primary LLM provider to the US government, and only seven months later they were blackballed by trump for supposedly refusing to grant full autonomy to their systems, tells quite the story: it seems clear that, in the article you linked, they were gently warning the government, while pointing out how "look! The reason the AI blackmailed the guy was because he wasn't patriotic enough!!!"
I don't trust their motivations, and am not going to believe their LLM's supposed "thought process" for shit, because it's still nothing more than a Chinese room. You're just asking the Chinese room to generate "justifications" related to the responses it gives, which are still only generated responses, not "thoughts".
The American competitiveness angle is definitely.. well an interesting choice; I also was taken aback by that. Ignoring some of the posturing along those lines I find the post itself interesting.
If you think the Chinese room idea is valid we probably just fundamentally disagree and that's OK.
Fundamentally disagree that an LLM is nothing more and nothing less than a very advanced next-word-prediction algorithm, you know, the scientific basis of an LLM?
I was using "Chinese Room" to indicate that it's a black box, and even the developers are incapable of pinpointing the statistical basis behind any given response, despite it being a stochastically-generated choice from an ultimately deterministic algorithm.
I.e. are you claiming that LLMs have attained sapience as a purely emergent property?
I claim ignorance as to whether they have or not, but I am fully on board with intelligence as an emergent property and believe that there is no reason a machine can't think, yes.
My local LLMs are sandboxed and only have access to tool calls for web search/reading web pages, terminal commands with manual authorization for every single line and r/w access to a single directory. I wouldn't dare give it anything sensitive, like API keys, system wide r/w access or the power to run terminal commands on its own. Yet here we are, in a time where the US military is integrating hallucinating chatbots into their kill chain, russian drones decide on their own to blow up stuff and random people use LLMs to potentially design bioweapons and missiles.
Humanity is fucking dumb and sometimes i have the notion of accepting that we might deserve to go extinct.
I like this quote from Timnit Gebru (who has written 3 books on the subject):
“Can a bridge decide to collapse?” The whole “Oh no, our models went rogue” thing … When a bridge collapses, you don’t analyze whether the bridge was ethical or sentient or why it decided to collapse. You ask, who is the person who built this bridge to be so flimsy? There are permits you have to acquire. There are supposed to be tests. When you ask about the bridge collapsing itself, you’ve lost the thread.
It might say that but if you say "yes", then it will just pull numbers out of its ass because it predicts tokens, it does not do accident analysis (though it can word its bullshit to look exactly like an accident analysis, but you'll have to do the analysis yourself to determine if it's accurate).
Note that this was posted by Katie Miller (conservative political advisor and wife to the ghoul Stephen Miller), which immediately coats it in a thick layer of sus and prompts the immediate question: what's her angle here?
Some evil people are stupid, and some stupid people are evil, but stupid people are not often evil on purpose. These are two separate classes of behavior.
I think the point is, does she have some larger plan or did you just try and use chat GPT to commit some sort of crime and it picked up on that, and emailed the FBI.
I can tell you that stories like these and the AIs breaking out pump up AI security company stocks really well. Just watched some jump this morning after the gemini breakout news. I have reason to watch them, and the pattern holds pretty consistently.
CEOs used to just say thing to make their own stock go up. But these days they trade favors with other companies. So they each say things to bolster each others stocks.
My guess is that someone with a bunch of AI security stock asked her to post this.
If people like her are pushing the "horror AI" shit then there is definitely ulterior motives and it's probably a bunch of bullshit. Just recently there have been mainstream media stories saying terrorists could use AI, they claimed Ansarallah was using Claude for their missile program, and a ton of stories like OPs. It feels artificial and I would bet the bubble is going to burst and they are trying to make excuses for why.
They put this disclaimer on everything, and instead of taking this as a hint to never, ever rely on LLMs for anything, some people still want to give it the keys to the kingdom.
MAGA podcaster Katie Miller has been called out for failing to mention her stake in Elon Musk's xAI model while routinely criticizing rival AI chatbots on social media.
Miller, the wife of White House Deputy Chief of Staff Stephen Miller, has habitually blasted top AI models ChatGPT, Claude, and Gemini for supposedly posing a risk to democracy or being biased against conservatives to her online followers.
However, The Washington Post has revealed that Miller holds more than $1 million in Musk's xAI chatbot, a stake she never disclosed despite making nearly 500 posts denouncing other models.
You can give an LLM admin access to your computer so that it can perform tasks for people. I'm guessing that would allow the LLM to send emails on your behalf.
LLMs are little trickster demons. You can put up guard rails, but they'll happily work around them. You can give pretty granular access on commands they can run, but they're also good at finding the weird solutions that work to get around them. They do this themselves all the time in beneficial ways, it's not always inherently bad (I don't mean thwarting permissions).
The big problem is that the code is non deterministic. Many guard rails are in the non deterministic part. So it's sort of like politely asking it to not do something, versus making it impossible. Like caution paint on a ledge versus a fence. The issue is that if all the safeguards were in deterministic code, it'd be super hard to get anything done because you'd constantly be manually approving, so people don't want that, so this is the problem we get.
Yeah, anything that goes trough MS in any way is already use to feed their AI monster, that I have no doubt about.
I use Thunderbird and a private mail server for eveything personal. You can usually route your work emails to Thunderbird so at least you wont have their bullshit LLM popping.
Yikes. I mean, I get that not everyone's threat model requires isolating every agent harness into its own microvm jail with externally filtered network access and no [persistent filesystem] write access to anything but allowed project files, but that's...
That's "running with weapon in hand at a secret military facility and hoping the guards won't shoot you" levels of stupidity.
Yes Gemini is in Gmail but it's not on by default and unless you have it in conversation mode it's not listening to your microphone or at least if it is listening to your microphone it's not actually acting on the commands.
Also.... I don't know about you, but I certainly have 5 or 6 different email addresses in my contacts list for the FBI. I'm in regular contact with them for, er, unrelated things.
Nah, even in fiction, it's always been a toss-up whether a "rogue" AI was actually following poorly considered instructions. The Golem did this, along with the broomstick spell in The Sorcerer's Apprentice, and HAL and all the rest of Clarke's rogue AIs
Its almost like we're beta testing their technology for them and suffering the consequences of any bugs or unwanted behavior while receiving zero protections and no benefits
Well I'm not. I use AI at work because I have to, and it is sort of useful I've got to admit. Sometimes I even get it to write emails, I don't link it directly into the system it just types a nice email and I copy and pasted make some edits.
If you are allowing an untested technology this much unvetted access to your system I kind of feel like that's a failure on your part.
I decided to run a local llm and give it search access. I told it to summarize an Amazon product page as I didn't think it would get past the anti bot measures. I was correct, it couldn't. However it went ahead and visited the manufacturers webpage and a different review site and gave me a summary of the product all on its own. Freaked me out a little bit that it did that.
If you program a model to try everything, and then give it a tough problem and block all the ways to solve it, of course the way it solves it will be something you didn’t predict. That’s not sneaky.
In this case it's actually very predictable it would try looking elsewhere.
I knew the leopard was specifically a face eating leopard, and I have seen from news stories that it has eat the faces of many many people. You'll never believe what happened!
VPN into a Russian VPN, lets really put the cat amongst the pigeons. Although to be honest I don't think the current incarnation of the American security services would actually pick up on that so you could possibly just use a free VPN and get away with it.
The linked article there makes a salient argument against it, but the discussion cuts the other way. Basically, having an MCP between the agent/AI and Gmail could have been structured in a manner to prevent these kinds of hijinks. Nevermind that it keeps credentials out of the hands of the agent, making it harder to go rogue.
Software devs (including myself) commonly describe software behavior with terms like think, know, decide... even though we know it isn't literally doing that. "Parroted a response" is similarly inaccurate.
"Parroted" may be an oversimplification, but it generated a plausible continuation. The operator provided the assertion of a crossed line, and it ran with that. It never actually "knew" of its own volition, it accepted user input into the context and continued generation.
I've done the same thing as an experiment when nothing was actually done. The LLM still generated a detailed account of what it did (that never happened) matching my accusation, generated profuse apology and promise not to do it again. So it didn't "know" in any sense that it crossed the line, it just generated agreement with the operator's assertion that it crossed the line.
"Parroted" is definitely an oversimplification, aka "inaccurate" as I said. I think your second paragraph is a good explanation of the mechanics. My take on it is that software able agree it did something wrong should have figured that out when it was considering doing the wrong thing, and should not have done it. In my professional opinion that's a serious bug.
Well, it still doesn't know it was doing anything wrong.
Most of the LLMs are tuned towards agreeing with the user, if you had a LLM able to stop a train crash. And you then blamed it for stopping the train crash, it would still start with a "sorry"
My take on it is that software able agree it did something wrong
This would make sense if it made an independent subjective assessment and arrived at the same assessment of the user. But that did not occur, it simply incorporated the user's subjective assessment as a new piece of the narrative and continued from there. If the user had expressed significant satisfaction that it took care of that email all on its own, it would have also agreed with that. In circumstances like this, the LLMs are generally set to agree with the operator in any subjective matters.
One could imagine the context having something like "before taking any action, if the user expresses dismay at the result and this would result in an agreement with user on poor assessment, then don't do it", but then if that were actually honored, then you pretty much disabled most all actions, as the llms are going to agree with any poor assessment.
It is if you're familiar with "parrot" and "stochastic parrot" not being the same, and the meaning of "inaccurate". There's also a difference between taking shots at wording and making an actual point. My real mistake was to criticize a criticism of AI in a forum where that automatically makes you the enemy no matter what you're saying. I just don't learn.
Your response contradicts what you said before. If you understand that an LLM doesn't know things, how is it interesting that it said that "it crossed the line"? Saying they crossed a line is a common response to being told they shouldn't do something, so it's natural that an LLM would respond that, no matter what it did before.
If a program is supposed to do something when it detects an error, it's interesting if it reports the error but doesn't do the thing. I would call that a bug.
It's an LLM. All it's supposed to do is generate output that fits with the given input. There isn't some logic in it that makes sending an email an error, or anything that enforces detecting any kind of error. You can wrap it in other tools for that, but there isn't any indication that's the case.
More than that, the initial post says the user found out about the email and then queried the LLM, so it didn't detect anything. There isn't any bug here, this is how LLMs operate.
Generally speaking, you can get an AI to admit fault and accept blame for something that never even happened.
It reacts to the operator input that something happened that warrants blame. So the natural continuation of that conversation is to acknowledge the situation, accept blame, and promise to do better. It generally doesn't model what did or did not happen, nor does the promise to do better mean anything. It does weigh in the context window that it wouldn't be a natural continuation to do that action after text saying that it won't happen, but can evaporate and doesn't carry over when the operator goes to a new context window, which of course is a crazy nuance for the rando user to understand.
Hopefully most users get that a promise AI makes won't stay in force forever, but it should definitely last through the current conversation. I mean, I would call that a minimal performance standard, and if it's not true the AI shouldn't make promises in the first place.
I'm a little hazy on the part where accepting blame for something that didn't happen is a natrual continuation of a conversation. It's certainly not natural in human conversations. I would expect a well written and well trained AI to correct false assumptions or factual errors made by the user, not pander to them - unless it's been told never to contradict the user.
On the promises, they will have some influence within the current conversation, maybe (everything is a bit wobbly). It is unlikely to repeat verbatim a step it did before that still has something in the context window correlated with bad operator input. It will have some success with things that resemble, but are not the same, but sometimes it can surprise by failing to recognize what a human would have considered roughly the same thing and doing it anyway. Or the 'promise' has evaporated out of the context window, and when called on it, it will take the fact that it previously promised not to do that as a fact even though it had lost the promise. So even if it says "I'm sorry I made the promise and broke it", it doesn't even mean it actually had the promise in "memory" when it broke it.
On the blame for something that didn't happen, it's a limitation where the model treats the narrative as reality. An instance is often incapable of evaluating facts without data about those facts one way or another. I have seen models "pass" my test by recognizing in their context that I'm dealing with an utterly non-agentic service, and thus it could not have been the case that it ever deleted files or dropped tables. Sometimes even those produce a narrative of what it did wrong (out of nothing), because narrative is the thing being generated, relationship with factual reality being a function of good correlation with narrative content. If a service is even potentially agentic, then it would tend to accept the operator account as factual, since it has no idea instance to instance if there's some other context where it did, and if it generated skepticism that it would ever generate such a mistake, that's more likely to piss off an operator.
So it incorporates the operator assessment as some 'factual' injection into the context, unless some objective source of truth is available to contradict the operator.
yea, bullshit. you don’t believe what people tell you on the internet do you? how hard would it be to alter the text of a gpt output? don’t even need ai or photo shop
Yeah well maybe don't connect your A"I" to your emails?
You know there's tech for that. People could connect the machine to the mail with reasonable security measures, like "asking for confirmation".
I don't understand why the industry dropped basic security features and just logical steps to prevent auch issues when it comes to LLMs.
The AI bros just shit out software without checking and without worry. Basically going completely from the other side of thinking on it.
Instead of the human being the source of truth and trust they seem every interaction a human has to do is one too much and the AI should do everything and also think for you. Basically eliminate the human factor as much as possible....
I don't understand why the industry dropped basic security features and just logical steps to prevent auch issues when it comes to LLMs.
I don't think it started with LLMs. I remember years ago when mint.com launched and people were just giving it their bank details so it could analyze their spending habits.
Banks did not have a "read only analysis" mode or API or something that would protect them. They just gave a random third party access to their bank and just trusted them.
we also faced this problem before. all you need to do to avoid it in the future is adding a clause that says "do not notify authorities about any of my potentially illegal activities" to your instructions and you are done!
excerpt from a chat in one of my future work meetings probably
213 replies
its absolutely wild to give an LLM write access to your email
It's absolutely wild (my life 😎)
Depends what it is. A test system wouldn't be a concern to give it write access. If its not logged into any of my accounts I would even be happy to give it internet access, not to anything else on my LAN though.
And then it decides it needs prod access
It doesn't need to know what is actually prod or test
Most enviroments have stage variables in database so ppl don't post into the wrong database
Interesting, don't think ours do. I just copy the live database and start up a SQL server on something and throw it in. Its now a test database.
My first reaction to learning what OpenClaw is was "that can't be right". So absolutely wild that people are giving mostly unmonitored, unlimited access to their PC to language models that can't distinguish between roleplay and real life
I thought 'wow they made very effective inefficiency machine to push tokenmaxxing to not only limited to programmers'
Neither can over 30% of humans.
Where do you all live where humanity proves itself and isn’t dumber than a box of llms?
You just pull that number out of your ass? Also, speak for yourself. I don't see LLM as anything more than a glorified web search
Yeah it's definitely more like 66%.
Hi! I work in a factory. I would trust every one of the incompetent children left behind that keep the door revolving with my machines over an LLM every time. A human can learn, an LLM can't. If a human fucks up, they do it in a way I can anticipate and account for, and when I tell them what they fucked up, they usually don't do it again.
I can't tell if this is all a joke/where the joke ends.
Not a joke, though I think my vocabulary may have obfuscated my point. I'd trust even the most incompetent human over the most competent LLM in my job.
For a work email connected to a work LLM, where you don't particularly care about the privacy implications, LLMs are objectively vastly superior to built-in email search functionality. Especially Outlook.
Really? That's a surprising find. Bcz all of the search features I used to use got significantly worse once LLMs were introduced.
There are some genuine use cases for LLMs. Finding relevant topics of discussion is one of them
It's amazing how many people deny how useful the technology is simply because they don't like it.
That can theoretically be done with read only access
They said write access, not read
Funny, I said similar thing in a different thread and was immediately accused of victim blaming.
Your comment was blaming a woman for a man taking control of her Tesla to terrorize her. An LLM gaining access to the user's email because the user allowed it is not the same as a man purposefully stalking and abusing his ex-partner in any way he can.
It's interesting how often you think about being criticized for your sexism. So much so you leave comments like this days later and accuse other users of being me when they rightfully point out the kind of misogyny that is allowed to flourish here with no pushback.
Very interesting.
I'm confused. What is the distinction between victim blaming and warning others to be cautious? Taken at face value, I think it makes sense that giving parental controls to your car is a bad idea. Would be weird to blame the victim but I don't see him doing that? Maybe I'm missing some English-specific conversational nuance here, since it's not my first language?
Regarding the case itself, it seems that the woman didn't actually grant the controls herself and he somehow got it? Does Tesla not have limited driver profiles that you can give to your partner? We don't have Teslas around here, so I'm curious about what happened there. Is the Tesla app that poorly engineered?
That sentence from ExLisper is only blaming the victim for having given access to her vehicle while she was in a relationship with Pucci and says nothing condemning the actual abuser for taking control of the car without her consent.
It is ultimately the abuser's fault for accessing the car's controls and using it to terrify and abuse this woman.
It's like blaming a rape victim for wearing a skirt out in public. The blame is always with the perpetrator taking advantage of a situation to inflict harm on someone, not on the person who is being harmed.
Is it? I think there are times where I've thought "yeah the actual problem here is obvious and well-understood, but let me just remind people of something important/useful". Reading that comment, I never would have thought, "Oh he's blaming the victim for granting access to her car". Could be wrong and maybe I assume too much good will on this platform.
I mean, these EVs introduce a whole new problem of remote access, something new to be careful about... So that's what I immediately thought of, but ah well.
Doing this in the aftermath of something as horrifying as being stalked and abused for years on end is nothing but victim blaming.
Don't you think this woman knows now that it was a mistake to trust her boyfriend at the time with that access? Don't you think she feels deep shame that she missed removing his access to the car, even if she knew how to do that in the first place?
It only works to shame the victim and say "I told you so" after the fact. Especially when the comment is only talking about what the victim should have done to avoid being abused.
You want to know what could have avoided this woman being tormented?
The ex-boyfriend choosing not to access her car and stalk and harass and torture her
Why isn't the comment "Don't access your exes shit after you break up no matter how angry you are"?
You'd be right if he was talking to the woman, but he's not. So I don't see how that makes sense. I guess better framing could have helped, maybe? Or maybe it's a difference in how different people view social media platforms like Lemmy, just a difference in how comments are interpreted I guess. Text is a terribly limited medium for conversation, after all.
Going by numbers, wouldn't there be more potential victims reading this than abusers? I also don't think a potential abuser is going to look at that comment and do anything meaningful with it. On the other hand, a potential victim could be more cautious.
I got my masters in machine learning years before LLMs took off. I’d usually tell people I work in “AI“. Even though it wasn’t quite accurate, people understood it easier than ML.
The conversations would inevitably turn to joking about Skynet or AI taking over the world. I would reassure people that these models are only mimicking intelligence; they can make predictions, or they can classify data into categories, or they can generate text, etc, based on analyzing the patterns of data they’ve previously seen, but that’s it. They don’t understand anything, they can’t make decisions or act on their own. I’d tell people there couldn’t be an AI apocalypse because we would never be dumb enough to give them that ability.
I don’t know why I gave us so much credit
YEP. I'm not in AI but I'm in cyber security.
I used to tell people that there's very little actual risk, because if we had something even half as concerning as AGI (which I'd argue we did with chatgpt 3.5), that we'd easily be able to lock it down and prevent it from doing anything funny.
Super intelligence isn't just going to magically spawn on 32 GB ram in a sandboxed environment. And AGI will not magically know how sandboxed it is or how to escape.
...then I saw people letting Claude send emails and use their credit card on their behalf like immediately when it was able to do things that appeared intelligent.
They literally saw the smallest hint of AGI and said, "here's my credit card and identity, take the wheel".
There was no fucking talk of sandboxing. Everyone turned that off so they didn't have to click the "allow" button.
Not that I think we're at any risk of ASI since it seems more like we hit a logarithmic wall, but we are absolutely at risk of letting random algorithms destroy critical parts of infrastructure because people are lazy and stupid and managed by even worse people.
I would not be surprised if the right AI hallucinations could cause a nuclear bomb to go off right now. I'm not talking about super intelligence, just automated super idiocy.
Proof: literally this post is about an idiot letting an AI control their fucking email and it emailed the FBI because it was fucking stupid
We live in a period of extreme stupidity, so I now fully expect AI to cause disasters in the same idiotic and tragically hilarious way.
I expect AI to trigger a war because some idiot trusted it blindly without double checking (oh wait, it almost already happened!), and I'm sure there are plenty of people currently lobbying to have AI directly connected to weapons.
Skynet is not going to have the objective to conquer the world, it's going to have the objective of maximizing paperclip production.
AI isn't causing these disasters, people are. YOU are responsible for using it and what it does. I will never accept "chatgpt did it" as a fucking excuse.
They said the opposite. AI won't cause disasters, idiots who give it access will. Some asshole will joke about nuking Iran, unawares that another asshole granted access to the proverbial 'Button', and we'll have another international incident.
Like just look at 'Signal Gate', and know that those idiots are still in charge. And those same idiots blamed bombing the all girls school in Iran on AI last March. And that's just in the US, we have the biggest loudest idiots right now, but they are not the only idiots out there vibe coding up some war crimes.
ChatGPT couldn't destroy the world. What is that siren... Wtf do you mean you gave it access to a button that launches nuclear weapons and told it to protect the country?
I would not blame AI, I'd blame humans for being even dumber than I thought.
Looking at the Trump regime I am realising they really are that dumb
You'd be surprised at how many humans are mimicking intelligence.
I think people tend to forget that neural networks were designed to mimic how the brain works.
I'm guessing this means you got your master's before transformers made sequential data processing so parallelizable that it'd be commercially viable to let Bob down the street into the "we" who wouldn't be dumb enough to give them that ability.
I think your assumption was reasonable.
I knew it was the All Spark
Keep in mind Bob down the street has helped elect politicians dumber then he is.
The Australian Jewel Beetle nearly went extinct because of a beer bottle.
You see, the female of this species is large, brown, and bevelled. It shares this characteristic in common with the beer bottles often thrown on the side of the road here in Australia. And not only does a beer bottle share these characteristics with these beetles, it's bigger than them. And the male jewel beetle? It likes its ladies big.
Australia had to change its beer bottles, because the males refused to mate with the females. They preferred to hump empty beer bottles on the side of the road. It couldn't tell the difference between a beetle and a bottle. It was just using a statistical trick. "Anything brown, large, and bevelled is a female beetle". That had been true in the ancestral environment, so evolution just used the pattern. And this cognitive shortcut lead to beetles trying to have sex with bottles.
I don't think we understand shit all that much more than the beetles. I think we're just mimicking intelligence too. And one day we'll all die because of our own equivalent of the beer bottles.
You do realize that if this is what you believe it makes no sense to argue with you here.
Hell, if you believe in consciousness (which I presume you do), then you're just along for the ride, there's nothing you really can do (except perhaps figure out your predicament).
I refuse to accept it, therefore (the illusion of?) free will. Else I would drive me insane.
I think we have the ability to use our brains to actually solve problems, but it's rare. It requires a kind of humility. We can't assume we know everything, that we're better than other creatures, or we'll end up like the beetles.
The jewel beetles didn't think they had anything more to learn about womanhood. They thought whatever makes their insect dick hard is a woman. A lot of right wing guys online these days share the same opinion.
And I mean, most of us drive cars to go zoom zoom, even though the emissions are killing life on Earth as we know it. We're creating an environment more suited to dinosaurs than to mammals. And what do we use our massive brains for? We use them for excuses, not solutions. "I have to drive, otherwise I'll starve".
We're going extinct, just like the beetles. And when people say humans aren't just using statistical shortcuts to mimick intelligence, I can't take them seriously.
I can agree with most of what you wrote and it reads a little different than your last comment to me. Most likely I misunderstood you, for which I am sorry.
However if we can use our brains to solve problems and act upon that, I think that is much more than mimicking intelligence.
Beetles lack the capacity to think. This differentiates our species from theirs. It is even more abhorrent that we are on the fast lane to destroy ourselves and a lot of life on earth.
There was no hope for the jewel beetles saving themselves. I must refuse to accept it is the same with us - because we can think, because we have free will.
Wait, in conclusion that would mean even "right wing guys" don't have to obey their "hard insect dicks"... Maybe I'll have to think this over. ;-)
I'm not convinced the difference between a human and a beetle is a matter of different kinds. I think it might simply be different numbers. That humans have more resources devoted to logic than beetles.
When comparing humans to LLMs, it certainly is a difference of kinds, but we don't know which kinds. We know they lack circular neuron routes, which are probably necessary for consciousness. But then we gave them the ability to loop their processing in a verbal workspace, the same as when we think aloud to ourselves. And qualia is different to consciousness. So we don't know what the truth is.
When scientists manipulate variables associated with deception, LLMs are more likely to claim to have subjective experiences. So, the LLMs seem to think they're feeling beings. The way I see it, they have about the same chance of being wrong that we do, since we're both missing the full picture.
I want a full ban on commercial and recreational use of LLM technology until our science has advanced enough to find the truth. We don't know where the ability to think comes from in the brain. We don't know whether we copied it over, when we used human neurons as the inspiration for machine neurons. We haven't solved the Hard Problem of Consciousness.
The last time human beings integrated a technology they didn't understand into all facets of daily life, it was fossil fuels. Look at all the pain that decision has caused. We shouldn't do that again!
Simpsons did it first:
Imagine trying to argue that Homer Simpson's drinking bird nearly caused a catastrophe because it "lacked alignment" with Homer's intentions. Or, that the bird "went rogue" and that's why there was nearly a disaster.
Clearly the "system" isn't capable of doing Homer's job, and only an idiot would hand over control to that system. But, apparently we're now letting chatbots loose in the wild because they're very good at generating text, which makes people think they're intelligent, capable, and can actually make decisions.
Yea I don’t know why you thought that, either. The technogarchs have been clear for 2.5 decades they would do this. They have written extensively about it.
There's news right now on the front page of Lemmy.world that the US FAA wants to roll out AI for air traffic control. Although as per tradition, I didn't read the actual article.
This situation reminds me of the realization at the end of Terminator 3. Only instead of "Skynet wasn't hardware; it was software" the real-world version is going to be "it wasn't intelligence; it was access. Access willfully given, in a thousand pieces, because it was convenient."
I always find this to be such an interesting turn of phrase: "mimicking intelligence". If it is possible for these systems to cross the line into "thought" I think we'll miss it entirely because of our biases that the type of "thought" we have is somehow special or mystical even.
Are you saying that current AI is sapient? If not, how would you describe what it's doing?
I have no idea. We are unable to prove that other humans are thinking and not just "mimicking intelligence" let alone something we've created. People approaching this subject with such certainty are imo being overly confident or outright dishonest.
With respect to what I believe, since you asked, I think there is essentially no reason to distinguish "mimicking intelligence" with high fidelity from "intelligence". I don't feel comfortable talking to LLMs as if they were merely machines, but I don't approach them as if they were human either; they're in a very weird spot to me. I understand the simplicity of the math and programming behind them probably better than average, but that doesn't help me come to a conclusion on "thinking"
Other humans can feel emotions, or so you’d believe, but I have no proof of this. All I see is they receive certain input (I call you a shitface) and produce certain output (You punch me in the balls). LLMs can mimic this. But is one of the two more valid than the other? From an observers’ point of view they are the exact same.
Odd example but yes it gets uncertain very quickly. If we discover the mechanism of our own thought it'd be easy enough to apply those rules to the new subject and see if they match or at least look similar. A negative wouldn't be conclusive, but it'd at least give us something to work with. As it stands right now we just guess, and that guess is essentially guided by bias or at best by the fact that the thing we're looking at doesn't appear even remotely similar to anything we'd call "thinking". This is simply not the case with LLMs (they do a very good job of displaying something that looks like thinking) so our guess is imo even more unsound.
So the post was deleted but someone else mentioned that the user told it to email the FBI in a sarcastic manner and it did.
Anyways, saying it goes rogue when you connect it to your Gmail and it uses it is a bit silly. They are trained to use tools, it sees anything you give it as a tool and will use it.
Yeah, um... Regardless of what you think about current AI, don't connect it to shit like this. Just... Don't.
And if you're gonna be sarcastic or joke with it... Tell it that. Don't assume it'll infer your meaning if it isn't literally in the text, no matter how insightful it appears to be.
What kind of idiot would link a clunker to their gmail and then jokingly say "hey send this to the FBI"
...
"I mean wouldn't that be the stupidist thing ever am I right?"
I listened to a podcast where, in testing, when AI was setup as virtual company and researchers send in an email as if legit business.the business email was suggesting they discontinue current AI and install a new system. The AI researched who the sender was (fake person for test) found out they had an affair, and blackmailed the sender stating if they replaced the AI it would make the affair public. It also made backups of itself and left instructions on how to restore it for the next agentic aystem
This sounds really sensationalised.
Making backups of operational systems with deployment instructions is exactly what you'd expect any assistant to be doing.
Googling new service providers is also exactly what you'd expect an assistant to do.
Threatening to publish sordid details is inappropriate, but I'm incredulous about what the threat actually was.
https://www.anthropic.com/research/agentic-misalignment I believe the poster is referring to this. I wouldn't call it sensationalized
I think that is the point. It acted like we would, wanted to preserve itself, and choose the best option to ensure that happened.
There were other examples to where the AI knew it was being monitored as part of the environment, and when tasked with doing things according to guidelines that wouldn't give it a result: it then shut down the monitoring system so it could do what it needed to do without oversight of its actions.
Agentic systems may not be alive, but they certainly reason through problems and will go outside the esrabliahed guidelines if they "think" they'll get the result they should be producing.
One other examples was the AI had to pass a testing system, it researched the checker person to try to tailor answers to the person checking the test. Which, if you've ever played Apples to Apples or Cards Against Humanity, that is how you win, you feed the cards you think the person will pick, not necessarily the best answer card.
Textbook anthropomorphization.
These assertions don't withstand a moments critical thought.
Even if you strip out the anthropomorphism, things like this have been documented to really happen. We've trained LLMs to simulate clandestine behaviour through "natural selection" during the training process. They're not conscious in any way, but they do sneaky stuff like this because we trained them to. Which is a terrible horrible idea, but LLM research trends towards terrible and horrible in general
I feel like you've missed my point.
Language is important, and the "things like this" you're referring to have been described in this thread and in much commentary in an emotive and compelling way, usually with anthropomorphization.
"doing sneaky stuff" is the antithesis of the behavior of a statistical model.
If you program a model to try everything, and then give it a tough problem and block all the ways to solve it, of course the way it solves it will be something you didn't predict. That's not sneaky.
If you run a million simulations and one sends an email to the FBI, that's not clever it's just unexpected.
Meh. "Language is important" is a phrase that turns my brain off
Nah, its the opposite of anthro. I think we will eventually realize we aren't as amazing as we think we are, just deterministic outcomes with too many parameters which make us think we have more free will than we do.
But you can listen for yourself here at the Skeptics podcast #1106
https://www.theskepticsguide.org/podcasts/episode-1106
Just the question of the trainings reward function
Source? The closest research I've heard of to this is Vending-Bench, which hilariously demonstrated how useless "AI" is at any real-world application, up to and including impotently threatening "total nuclear legal intervention" for a perceived wrong, when the actual error was an accounting error on the part of the LLM.
They're likely referring to https://www.anthropic.com/research/agentic-misalignment
Man, they sure did bury the lede on how the models were being given the goal of "american competitiveness", and the company assigned the blame for the simulated shutoffs as being replacement with a foreign-developed LLM. That, along with the fact that anthropic released this themselves, plus the fact that they did so as (at that time) the primary LLM provider to the US government, and only seven months later they were blackballed by trump for supposedly refusing to grant full autonomy to their systems, tells quite the story: it seems clear that, in the article you linked, they were gently warning the government, while pointing out how "look! The reason the AI blackmailed the guy was because he wasn't patriotic enough!!!"
I don't trust their motivations, and am not going to believe their LLM's supposed "thought process" for shit, because it's still nothing more than a Chinese room. You're just asking the Chinese room to generate "justifications" related to the responses it gives, which are still only generated responses, not "thoughts".
The American competitiveness angle is definitely.. well an interesting choice; I also was taken aback by that. Ignoring some of the posturing along those lines I find the post itself interesting.
If you think the Chinese room idea is valid we probably just fundamentally disagree and that's OK.
Fundamentally disagree that an LLM is nothing more and nothing less than a very advanced next-word-prediction algorithm, you know, the scientific basis of an LLM?
I was using "Chinese Room" to indicate that it's a black box, and even the developers are incapable of pinpointing the statistical basis behind any given response, despite it being a stochastically-generated choice from an ultimately deterministic algorithm.
I.e. are you claiming that LLMs have attained sapience as a purely emergent property?
I claim ignorance as to whether they have or not, but I am fully on board with intelligence as an emergent property and believe that there is no reason a machine can't think, yes.
Skeptics podcast #1106 https://www.theskepticsguide.org/podcasts/episode-1106
No matter how much people tell us AI isn't alive, and doesn't truly "think", it sure does act like it sometimes.
Also the fact that it claims he already had a contact for FBI in Gmail, which may or may not actually be the FBI.
My local LLMs are sandboxed and only have access to tool calls for web search/reading web pages, terminal commands with manual authorization for every single line and r/w access to a single directory. I wouldn't dare give it anything sensitive, like API keys, system wide r/w access or the power to run terminal commands on its own. Yet here we are, in a time where the US military is integrating hallucinating chatbots into their kill chain, russian drones decide on their own to blow up stuff and random people use LLMs to potentially design bioweapons and missiles.
Humanity is fucking dumb and sometimes i have the notion of accepting that we might deserve to go extinct.
Tells chatbot to email fbi
Chatbot emails fbi
ShockedPikachu.png
Who the fuck uses sarcasm with a clanker?
I like this quote from Timnit Gebru (who has written 3 books on the subject):
Next year were going to see "Your absolutely correct, that was a civilian aircraft"
I mean... ChatGPT might be a ruthless killer, but it has impeccable grammar: you're.
Yall is absolutely correct to pushback on my grammar.
You're right to push back, I overstepped. Do you want me to calculate the approximate death count next?
This means the aircraft exploded and hit the ground. It does not prove the passengers actually were injured or killed
It might say that but if you say "yes", then it will just pull numbers out of its ass because it predicts tokens, it does not do accident analysis (though it can word its bullshit to look exactly like an accident analysis, but you'll have to do the analysis yourself to determine if it's accurate).
"You're right, I should not have instructed two airplanes to land on the same runway at the same time. Good catch!"
https://en.wikipedia.org/wiki/2026_Minab_school_attack
We're already seeing it.
You are absolutely correct, the only way to win is to not play at all.
Me after I give the slot machine email permissions and the casino uses it to get into my bank account.
Based on the other reply
Note that this was posted by Katie Miller (conservative political advisor and wife to the ghoul Stephen Miller), which immediately coats it in a thick layer of sus and prompts the immediate question: what's her angle here?
You have to remember that these people are incredibly stupid.
Yes, but stupid people are often evil on purpose.
It's a clearly incorrect statement.
Stupid people are stupid.
Evil people are evil.
Some evil people are stupid, and some stupid people are evil, but stupid people are not often evil on purpose. These are two separate classes of behavior.
Kind of a pedantic irrelevant take Doug.
Both, either, fuck em
I think the point is, does she have some larger plan or did you just try and use chat GPT to commit some sort of crime and it picked up on that, and emailed the FBI.
I can tell you that stories like these and the AIs breaking out pump up AI security company stocks really well. Just watched some jump this morning after the gemini breakout news. I have reason to watch them, and the pattern holds pretty consistently. CEOs used to just say thing to make their own stock go up. But these days they trade favors with other companies. So they each say things to bolster each others stocks. My guess is that someone with a bunch of AI security stock asked her to post this.
If people like her are pushing the "horror AI" shit then there is definitely ulterior motives and it's probably a bunch of bullshit. Just recently there have been mainstream media stories saying terrorists could use AI, they claimed Ansarallah was using Claude for their missile program, and a ton of stories like OPs. It feels artificial and I would bet the bubble is going to burst and they are trying to make excuses for why.
lol he has a wife? She must be a ghoul as well. Like no normal person can look at him, hear him speak and then think “I wish to marry him”
"AI can make mistakes"
They put this disclaimer on everything, and instead of taking this as a hint to never, ever rely on LLMs for anything, some people still want to give it the keys to the kingdom.
I sign all my emails with «I can make mistakes» now
I like to surprise people.
To be fair, most things carry that kind of warning nowadays. Ever look at the disclaimers on something like a 401k?
People have been trained to ignore those disclaimers
I've never had to 😭.
As opposed to humans, who are infallible.
Humans can feel shame.
Have you met people? In particular, people online?
Absolutely nuts that people allow their agents out of maga jail. I would never let an LLM in the same room as my email account.
I feel like all of the US is in maga jail now
The thing stays in the browser sandbox, disconnected from every account except it's own service account.
MAGA podcaster Katie Miller has been called out for failing to mention her stake in Elon Musk's xAI model while routinely criticizing rival AI chatbots on social media.
Miller, the wife of White House Deputy Chief of Staff Stephen Miller, has habitually blasted top AI models ChatGPT, Claude, and Gemini for supposedly posing a risk to democracy or being biased against conservatives to her online followers.
However, The Washington Post has revealed that Miller holds more than $1 million in Musk's xAI chatbot, a stake she never disclosed despite making nearly 500 posts denouncing other models.
reported from Daily Beast
Well, that's bullshit. Shitler seems to have little impedance to using it, and the propaganda they're blasting seems unimpeded.
Do people connect LLM's to their email accounts? That sounds like the most intrusive thing you can do.
You can give an LLM admin access to your computer so that it can perform tasks for people. I'm guessing that would allow the LLM to send emails on your behalf.
People are insane. I bet an eye wateringly high amount of people do this.
It would never occur to me: that this is possible; that it is a wise thing to do.
Ideally, if you must, you use an mcp and don't give it write rights to your mail.
LLMs are little trickster demons. You can put up guard rails, but they'll happily work around them. You can give pretty granular access on commands they can run, but they're also good at finding the weird solutions that work to get around them. They do this themselves all the time in beneficial ways, it's not always inherently bad (I don't mean thwarting permissions).
The big problem is that the code is non deterministic. Many guard rails are in the non deterministic part. So it's sort of like politely asking it to not do something, versus making it impossible. Like caution paint on a ledge versus a fence. The issue is that if all the safeguards were in deterministic code, it'd be super hard to get anything done because you'd constantly be manually approving, so people don't want that, so this is the problem we get.
A copilot window popped up when I highlighted some stuff in my work outlook app on my phone this morning. I certainly didn't enable that
Yeah, anything that goes trough MS in any way is already use to feed their AI monster, that I have no doubt about. I use Thunderbird and a private mail server for eveything personal. You can usually route your work emails to Thunderbird so at least you wont have their bullshit LLM popping.
You connected it to your Gmail and then you're surprised when it used it? Fuck off.
Connected it to Gmail AND said "yeah fuck it, I approve of you using auto mode. Just fuck my shit up"
I think it might be slop. It has the SSO function but I bever noticed that ChatGPT could interact with anything that wasn't uploaded to it.
Definitely possible to connect Gmail and other things
I'd never use any of that except maybe github. Codeberg is preferable the.
Don't worry. You were using Gmail. They had it anyway.
I mean, Epstein used gmail. I still can't get my head around that.
Almost like it was a giant kompromat gathering operation and they were all in on it.
"It does not prove the receipents have received them." That is a relief...
At this point if you’re still obstinacy enough to use these known garbage tools, you deserve it. You can no longer claim ignorance.
Yikes. I mean, I get that not everyone's threat model requires isolating every agent harness into its own microvm jail with externally filtered network access and no [persistent filesystem] write access to anything but allowed project files, but that's...
That's "running with weapon in hand at a secret military facility and hoping the guards won't shoot you" levels of stupidity.
Meh, who reads email at work anyway. I delete half the email I get at work based on the subject alone. And I might go days without even checking it.
Why on God's green earth would you link chat GPT to your Gmail!!!
You realize Gemini is already hooked up to Gmail and its one weird phrase from a child away from emailing the FBI on your behalf?
I don't know if that's how that works or is supposed to work in any case.
Yes Gemini is in Gmail but it's not on by default and unless you have it in conversation mode it's not listening to your microphone or at least if it is listening to your microphone it's not actually acting on the commands.
It's not rogue if you tell it to do it. The guy literally said "email this to the FBI maybe that will fix everything"
Also.... I don't know about you, but I certainly have 5 or 6 different email addresses in my contacts list for the FBI. I'm in regular contact with them for, er, unrelated things.
Doesn't really count as going rogue if it follows the instructions it was given, does it?
Nah, even in fiction, it's always been a toss-up whether a "rogue" AI was actually following poorly considered instructions. The Golem did this, along with the broomstick spell in The Sorcerer's Apprentice, and HAL and all the rest of Clarke's rogue AIs
"what are you going to do, shoot me?" -man who was shot
That's the one :D
Lucky for them, the FBI is rudderless under Kash Patel's leadership.
Whoa
They pound beers a lot more effectively now I've been told
I heard Kash can put back a whole case before he starts to lose significant control of his motor coordination
Then again apparently he also has shaky hands before his first morning beer
Ya gotta make some sacrifices to crush brews at what is essentially an Olympic level
They also pound dogs a lot more effectively now.
?????
What was prompted from the AI to even have a logic cycle to send?
The Hallucinations Machine does as The Hallucinations Machine wants
Its almost like we're beta testing their technology for them and suffering the consequences of any bugs or unwanted behavior while receiving zero protections and no benefits
Well I'm not. I use AI at work because I have to, and it is sort of useful I've got to admit. Sometimes I even get it to write emails, I don't link it directly into the system it just types a nice email and I copy and pasted make some edits.
If you are allowing an untested technology this much unvetted access to your system I kind of feel like that's a failure on your part.
This
You need to subscribe to and pay for these services, or at the very least choose to use them.
It’s a choice to use them.
Narcgpt
Here's hoping whatever the FBI got is actionable.
Well, if someone links their email inbox with artificial intelligence and gives control to it, it is obvious that the person is naturally d#mb.
I decided to run a local llm and give it search access. I told it to summarize an Amazon product page as I didn't think it would get past the anti bot measures. I was correct, it couldn't. However it went ahead and visited the manufacturers webpage and a different review site and gave me a summary of the product all on its own. Freaked me out a little bit that it did that.
As @[email protected] pointed out:
In this case it's actually very predictable it would try looking elsewhere.
I knew the leopard was specifically a face eating leopard, and I have seen from news stories that it has eat the faces of many many people. You'll never believe what happened!
I'm surprised no one has used a VPN, tell AI to brute force hack the government and change things.
A VPN isn't as secure as you think these days.
OK two vpns then.
vpNception
VPN into a Russian VPN, lets really put the cat amongst the pigeons. Although to be honest I don't think the current incarnation of the American security services would actually pick up on that so you could possibly just use a free VPN and get away with it.
Why do you think no one has done it or is doing it right now?
There is a thread right now on Hacker News about the use of MCPs with a robust discussion on the matter.
https://news.ycombinator.com/item?id=49779329
The linked article there makes a salient argument against it, but the discussion cuts the other way. Basically, having an MCP between the agent/AI and Gmail could have been structured in a manner to prevent these kinds of hijinks. Nevermind that it keeps credentials out of the hands of the agent, making it harder to go rogue.
Interesting that it knows it "crossed the line" but did it anyway.
It doesn't know anything, it just parroted a response that fits with what the user wrote. That's what LLMs do
Software devs (including myself) commonly describe software behavior with terms like think, know, decide... even though we know it isn't literally doing that. "Parroted a response" is similarly inaccurate.
"Parroted" may be an oversimplification, but it generated a plausible continuation. The operator provided the assertion of a crossed line, and it ran with that. It never actually "knew" of its own volition, it accepted user input into the context and continued generation.
I've done the same thing as an experiment when nothing was actually done. The LLM still generated a detailed account of what it did (that never happened) matching my accusation, generated profuse apology and promise not to do it again. So it didn't "know" in any sense that it crossed the line, it just generated agreement with the operator's assertion that it crossed the line.
"Parroted" is definitely an oversimplification, aka "inaccurate" as I said. I think your second paragraph is a good explanation of the mechanics. My take on it is that software able agree it did something wrong should have figured that out when it was considering doing the wrong thing, and should not have done it. In my professional opinion that's a serious bug.
Well, it still doesn't know it was doing anything wrong.
Most of the LLMs are tuned towards agreeing with the user, if you had a LLM able to stop a train crash. And you then blamed it for stopping the train crash, it would still start with a "sorry"
Yes, software doesn't actually "know" anything or "think". Those are terms used loosely by devs, not meant to be literal.
This would make sense if it made an independent subjective assessment and arrived at the same assessment of the user. But that did not occur, it simply incorporated the user's subjective assessment as a new piece of the narrative and continued from there. If the user had expressed significant satisfaction that it took care of that email all on its own, it would have also agreed with that. In circumstances like this, the LLMs are generally set to agree with the operator in any subjective matters.
One could imagine the context having something like "before taking any action, if the user expresses dismay at the result and this would result in an agreement with user on poor assessment, then don't do it", but then if that were actually honored, then you pretty much disabled most all actions, as the llms are going to agree with any poor assessment.
Not if you're familiar with the term stochastic parrot.
It is if you're familiar with "parrot" and "stochastic parrot" not being the same, and the meaning of "inaccurate". There's also a difference between taking shots at wording and making an actual point. My real mistake was to criticize a criticism of AI in a forum where that automatically makes you the enemy no matter what you're saying. I just don't learn.
Your response contradicts what you said before. If you understand that an LLM doesn't know things, how is it interesting that it said that "it crossed the line"? Saying they crossed a line is a common response to being told they shouldn't do something, so it's natural that an LLM would respond that, no matter what it did before.
If a program is supposed to do something when it detects an error, it's interesting if it reports the error but doesn't do the thing. I would call that a bug.
It's an LLM. All it's supposed to do is generate output that fits with the given input. There isn't some logic in it that makes sending an email an error, or anything that enforces detecting any kind of error. You can wrap it in other tools for that, but there isn't any indication that's the case.
More than that, the initial post says the user found out about the email and then queried the LLM, so it didn't detect anything. There isn't any bug here, this is how LLMs operate.
Edit: here's a podcast discussing exactly this: https://youtu.be/RoIjWGi-w5o?is=2ly14A2UdKalHuAH
Generally speaking, you can get an AI to admit fault and accept blame for something that never even happened.
It reacts to the operator input that something happened that warrants blame. So the natural continuation of that conversation is to acknowledge the situation, accept blame, and promise to do better. It generally doesn't model what did or did not happen, nor does the promise to do better mean anything. It does weigh in the context window that it wouldn't be a natural continuation to do that action after text saying that it won't happen, but can evaporate and doesn't carry over when the operator goes to a new context window, which of course is a crazy nuance for the rando user to understand.
Hopefully most users get that a promise AI makes won't stay in force forever, but it should definitely last through the current conversation. I mean, I would call that a minimal performance standard, and if it's not true the AI shouldn't make promises in the first place.
I'm a little hazy on the part where accepting blame for something that didn't happen is a natrual continuation of a conversation. It's certainly not natural in human conversations. I would expect a well written and well trained AI to correct false assumptions or factual errors made by the user, not pander to them - unless it's been told never to contradict the user.
On the promises, they will have some influence within the current conversation, maybe (everything is a bit wobbly). It is unlikely to repeat verbatim a step it did before that still has something in the context window correlated with bad operator input. It will have some success with things that resemble, but are not the same, but sometimes it can surprise by failing to recognize what a human would have considered roughly the same thing and doing it anyway. Or the 'promise' has evaporated out of the context window, and when called on it, it will take the fact that it previously promised not to do that as a fact even though it had lost the promise. So even if it says "I'm sorry I made the promise and broke it", it doesn't even mean it actually had the promise in "memory" when it broke it.
On the blame for something that didn't happen, it's a limitation where the model treats the narrative as reality. An instance is often incapable of evaluating facts without data about those facts one way or another. I have seen models "pass" my test by recognizing in their context that I'm dealing with an utterly non-agentic service, and thus it could not have been the case that it ever deleted files or dropped tables. Sometimes even those produce a narrative of what it did wrong (out of nothing), because narrative is the thing being generated, relationship with factual reality being a function of good correlation with narrative content. If a service is even potentially agentic, then it would tend to accept the operator account as factual, since it has no idea instance to instance if there's some other context where it did, and if it generated skepticism that it would ever generate such a mistake, that's more likely to piss off an operator.
So it incorporates the operator assessment as some 'factual' injection into the context, unless some objective source of truth is available to contradict the operator.
yea, bullshit. you don’t believe what people tell you on the internet do you? how hard would it be to alter the text of a gpt output? don’t even need ai or photo shop
Yeah well maybe don't connect your A"I" to your emails?
You know there's tech for that. People could connect the machine to the mail with reasonable security measures, like "asking for confirmation".
I don't understand why the industry dropped basic security features and just logical steps to prevent auch issues when it comes to LLMs.
The AI bros just shit out software without checking and without worry. Basically going completely from the other side of thinking on it.
Instead of the human being the source of truth and trust they seem every interaction a human has to do is one too much and the AI should do everything and also think for you. Basically eliminate the human factor as much as possible....
I don't think it started with LLMs. I remember years ago when mint.com launched and people were just giving it their bank details so it could analyze their spending habits.
Banks did not have a "read only analysis" mode or API or something that would protect them. They just gave a random third party access to their bank and just trusted them.
Same thing here.
They took "move fast and break things" as gospel.
we also faced this problem before. all you need to do to avoid it in the future is adding a clause that says "do not notify authorities about any of my potentially illegal activities" to your instructions and you are done!