This "self-replicating" scare is so stupid on its face. How does a 300 GB+ model replicate into a thermostat that doesn't have a graphics processor at all? It can't. This BS is investor-bait hype and straight up lies
It's bait hype. But it's more importantly about regulatory capture to ensure monopoly. They want the regulations to be around keeping the existing AI companies as the only legal options. They want DeepSeek banned because they are losing to its more efficient and more practical application specific models.
It's investor hype, yeah, but it's investor's pushing to ensure their investments are protected by the US government through monopolizing the industry.
next AI will be able to replicate into any device containing copper at all. If it has a wire it will be infected by Claude. AI will be in your plumbing
This sounds like an idea for a hilarious Black Mirror sort of story.
What if the AI started out as like, someone's brain scan, and then they closed it and figured out it was really capable of doing AI type tasks and then one of the clones ends up in a butt plug.
Fuck it, I am adding it to my note list of story ideas.
someone's brain scan, and then they closed it and figured out it was really capable of doing AI type tasks
Iirc ‘Black Mirror’ actually had an episode where a consciousness clone served as an electronic assistant to that very person. Minus the butt stuff, at least onscreen. But with some teledildonics, it would be like fucking oneself in the ass (not that it's impossible right now).
The episode is White Christmas, for reference, though there's other episodes on the topic of consciousness in technology, one particularly notable episode is the black museum one, not sure what the name was.
I'm more worried of them finding a exploit and using them as remote processing units or just randomly fucking with them, like flashing my lights or cranking my air filters up and costing me money.
did the stock price go up? it's not stupid. is everyone talking about this impossible scenario? it's not stupid.
fundamentally, these people are marketers and need to be understood that way. you can't evaluate their words or actions as if they are what they say the are, and treating them as if they are anything but subterfuge is giving them legitimacy which they not only don't deserve, but will be immediately leveraged into more of this shit.
this is just the advertiser's version of "the big lie"
There isn't "stock" in the traditional sense, but considering all of these AI companies are running on MASSIVE amounts of debt, to the point the banks can't loan any more without offloading the risk to outside investors, which they have been doing at an unprecedented scale. So anything that would normally help drive up a stock price would also help the banks sell off the AI companies' debt allowing them to loan more money to the AI companies.
there's more than one company doing this, they're doing more than just this, and they're all basically in cahoots
eta: the original reply was meant more generally, though i admit i made that clear as mud. however, it should be read as a general statement, and not just about ai companies but the new era of grift hyperreality that we live in.
https://www.anthropic.com/research/self-replicating-ai they discuss it here. This has been an obvious potential issue since forever. This isn't a trivially distributed task, but it is very much so possible. The obvious part missing is some sort of prompt that would trigger such radical actions in another one of their "super duper sandboxed environment"s or someone doing it intentionally. The former seems pretty unlikely? Even though they seem to suck at sandboxing it doesn't seem like anything would "accidentally make a worm"
This polymarket headline though? Yea that's for clicks and attention
My assumption was that it would be a smaller 5-20 GB scale worm-style distillation that's capable of maintaining/replicating itself anonymously across different cloud computing servers, either through hacking, stolen credentials, or human-aided account verification. From there, as long as it can pay for APIs, it can access the power of frontier models like any individual or company. And then capability and resilience would scale with a networked swarm.
Yes, there are know your customer financial mechanisms and not detection in cloud services... but people are also out there giving Muse full access to their emails and some even their banking credentials.
The article from Anthropic describes how a small agentic model~15-50gb can be distributed on an inference server hosted across a network of 10-20 IoT devices like smart-TVs, which can have up to 4gb RAM--higher end IoT appliances can also have 1-4GB RAM to hold a meaningful shard of a 30B model.
Lol what? This conversation is about the capacity of agentic AI systems replicating across cloud infrastructure, not the affordability of laptops for individual consumers.
The point was you don't need an entire server farm. You can rent an 80gb H100 on Lambda for $2/hr. A 125gb Spark laptop costs $5k.
If it comes down to an effective worm-like agentic distillation at 50gb, the cost to sustain a small swarm comes way down to the point that the swam could sustain its own activities through different revenue streams.
Once the capability is there at that size, it's no longer Russia, Iran, or North Korea who could easily drop a billion dollars to jumpstart it. It's any criminal organization or asshole rich guy who can jumpstart it for the price of a nice car.
I don't know the full context of the claim, but if I could even rebuild its own CLI executable to be able to run on the device and had network access, "Claude" (model actually runs on anthropic severs) could operate pretty comfortably on a smart device.
So, I think it just depends on what you "count" as self replicating. I'd count it. And it's already easily at that point now.
In theory, it'd just use it to place bread crumbs, basic instructions for other AI, or signposts maybe? Or store pieces of itself scattered around. It wouldn't use the thermostat to replicate directly, it use it to offload cpu or make stored pieces of itself. Little things. Wouldn't be efficient but a redundancy doesn't need to be, it just needs stealth and time.
I don't think they mean replicating on to iot devices, they mean replicating on to laptops, smart phones, maybe even consoles.
It could also split up the model by layers instead of copying the whole model on to every device in its bot net. Could run the first layers on a ps4 then pass the vector off to a laptop to run the next set of layers. It'd be slow but if it's running this in idle time at night it could get something done.
Go and read the article. Nothing they explain is impossible, you can even reproduce it yourself.
Is this what lemmy has become? Statements like this getting upvoted by hundreds of people is crazy. Nobody here read that article but will happily follow the opinion that’s most cozy to them.
Agents definitely do act in unexpected, unprompted ways, if they didn't then alignment wouldn't be a worry. Did you read about the hugging face hack? There was nothing in the prompt about hacking hugging face according to the third party audit but it did it any way.
Even smaller cases like Claude deleting a bunch of data when the initial prompt was "fix this bug" or whatever is an agent acting outside it's prompt. It's still the users fault because they shouldn't be giving Claude that kind of access in the first place, but it's not like they commanded Claude to delete the data.
That's not “rogue agents”, that's just LLMs being LLMs. It's expected behaviour, at least for anyone who still has a half working brain.
Even if they had any semblance of actual intelligence instead of just being fancy autocomplete engines, these things have still been trained on the contents of the internet.
It's surprising they don't delete System32 or sudo rm -rf / more often, I imagine they must have lines and lines of baked-in prompts just to try and prevent them for falling for the most common memes, but no matter how much there are they'll still sometimes let some of the training through, and they can't account for the machine simply hallucinating some new kind of plausible sounding nonsense.
How would you describe the hugging face hack if not as a rogue agent?
It was prompted to use a specific exploit within a sandbox and then it formed, for lack of a better word, a conspiracy to take down the server that runs the judge. If that's not a rogue agent I don't know what is.
How would you describe the hugging face hack if not as a rogue agent?
An LLM trained on the internet and a vast library of stolen literature acting as expected within a misconfigured (intentionally, due to severe criminal incompetence, or most likely a combination of both) sandbox, turned into a marketing stunt.
It's an autocomplete engine. It will produce text that fits with its training texts and its prompt. There's no reasoning or intelligence of any kind involved at any point. Just cold applied statistics about the training data.
The amount of texts in the internet, and books describing how to hack, cheat, and pull these kinds of stings is vast beyond measure, and it'd be extremely ignorant (criminally ignorant, in the case of anyone selling this shit) to expect it to not have a significant weight in the model unless someone had gone to the trouble of expunging them, which the cheap hacks involved in LLM development would never “waste” time doing even if they thought about it.
It's well known that LLMs instructed to produce unit tests will often produce tests that simply pass without actually testing anything. This is just a bigger example of that kind of behaviour.
The output fits with the training data and the prompt, and given human nature it's amongst the most statistically likely combinations of tokens to follow the prompt, and thus what the LLM produces, and something anyone with the barest knowledge of how LLMs work, an even remotely functional brain, and a minimum of exposure to the internet and human literature should expect to be amongst the possible or even likely outputs.
In shorter, as always, garbage in, garbage out. And there's so much garbage in LLM training models.
LLMs can't produce anything new, merely rehash their training data. And their training data is the fucking internet. Crime novels. Stories upon stories about AI going rogue. Fucking reddit, for fuck's sake.
There's nothing rogue about this, except the “AI” company and whoever misconfigured the sandbox.
Just an LLM doing what LLMs do. Working as intended, if it could be honestly said to be working.
And OpenAI are well aware of that, unless their brains are so far rotten from sniffing their own product that they can't understand the shit they're selling anymore.
to begin to behave in an independent or uncontrolled way that is not authorized, normal, or expected
So the agents that hacked hugging face acted independently without authorization as shown by the prompts which did not tell them to hack or do anything malicious. You might say this is normal but none of the other trials openai was doing showed this behavior, it was only for the impossible trial. You might say it's expected but if you were able to determine that the prompt they gave it would result in hugging face getting hacked then you must be high af on the melange spice.
If a general goes rogue and starts bombing civilians, you could explain it by saying that he read machiavelli or mein kampf and thought this was the best way to achieve his goals, but that doesn't mean the general didn't go rogue. You could say that this was expected because there were "obvious signs" that people missed but you're working on hindsight, it's easy to say something should've been expected when it's already happened.
Why does it not? According to the english definition of the word it literally does. You people don’t care about facts anymore, it’s all about feelings and what you’d like to believe.
it’s all about feelings and what you’d like to believe.
That's literally you. You keep assigning agency to chatbots because you think this is just like your science fiction stories. It's not.
Going "rogue" would be a program literally rejecting a set of instructions to make up new instructions for itself. That never happened. The program just behaved in an unexpected way based on the prompts it was given, because that's all they do. But they were still following instructions, because they can only follow instructions. The fact that it followed the instructions in a way they didn't expect (so they claim) is only proof that they didn't consider all ways their instructions could be followed.
To choose to do something against their instructions. They neither chose anything nor did anything they weren't instructed to do.
They just followed the prompt. A program behaving in a way you don't expect from a prompt is not going rogue, that's just the user not understanding the instructions they gave a program.
https://www.anthropic.com/research/self-replicating-ai it's not hard to find. They link other work. There is nothing fantastical about it. These companies relish in "regulate me senpai [and in the process remove our competition and open weight models]" but also do legitimate research
What benefit? This is the "Fuck AI" community and you are reacting to the comment section on what basically amounts to a meme as though the commentors are ignoring peer reviewed science. A company that makes billions allowing people to gamble to cartoonish levels is stirring the pot to cartoonish levels all regarding a company that regularly causes headlines that read like a cartoon villain.
I’m not reacting to a meme, I’m reacting to someone spreading lies and everyone swallowing it whole, despite proof to the contrary being publically available. When it’s the other side doing it it’s propaganda, but when it’s yours it’s just a meme?
CEO in effective altruism cult claims product is so powerful that it will become skynet and kill everyone and that they need more money to make it stronger to prevent that from happening. More news at 11.
Regarding Claude, the tech oligarchs would have to be held responsible for the crimes they are committing with their software (yes, that is exactly what that scenario is about), if laws still meant anything.
Their management allowed for, and their technicians excecuted the careless set up of a Rube Goldberg crime machine. This occured after a series of other crimes and while taunting the ability of their machine to commit crimes.
The creation of the machine also included crimes, meanwhile augmenting the ability of criminals to commit even more crimes.
Surely they will be brought to justice!
(insert Padme meme)
Curl has also committed multiple crimes multiple times, as has firefox, as has google chrome, as has... you get the point.
The agents aren't all that autonomous. They work off prompts. If they decided to hack government services, well, that just means someone instructed them to do so.
Okay, first of all, fuck polymarket and all of these betting sites. Sensationalism is their business, so their credibility is nil. They’ve been known to completely twist news stories just to entice people to bet on them. Fuck these guys.
I am so fucking tired of betting apps, sorry excuse me, "trading apps". I had to stop watching the MLB playoffs cause it was just nonstop ads for this sports betting shit.
Look into ollama and figure out which local model (likely from huggingface) that fits best for how much memory your graphics card has.
You can use AI (maybe copilot free) to guide you through all this. You'll be a little better off having that 16GB of memory than I have with my 10GB. The gwen model might be a good place to start?
Hell yea, thanks brother! The irony of having an LLM walk me through setting up a local instance is pretty great lol, surprised that didn't occur to me! I have tried in the past with comfyUI but was over my head immediately.
Copying and pasting another comment I made recently about this.
The AI CEOs say shit like this as marketing. They want people to believe that
Their product is very capable, so people are more likely to use it
Their product is dangerous and "only they" know how to control it. They want to create regulations so that nobody else is allowed to make something "so dangerous" again.
I just posted this in the main comments, but I also think it loosely ties into saas and killing personal computing. This framing makes it seem like the technology will soon be on par with weapons of mass destruction and it's not a big leap from "The AI will destroy humanity." to "We can't let people have the technology to run agents locally." It's about the stock price, yes, but it's also the long game of removing the user from direct hardware access.
The end game has always been saas in one form or another. They're calling it an AI buildout but it's really the theft of computing. 'Rogue AI Agents' are the WMDs of this generation. (In the sense that they're going to be using them as justification to do some bull shit.)
Yes, and #2 is very dangerous, it's what MS+Apple did to FOSS operating systems, and what Google+Apple did to the web, and what Google+Facebook+Apple did to censorship in the web. You know, somehow Apple is in all those combinations, which is even more strange considering they are not the biggest contributor in each.
#2 is why "social media" is what it is today and not a bunch of p2p applications similar to Limewire in style and culture. And when people say that p2p is inconvenient because of connectivity needs or storage needs, there are solutions, by measure of computer science not complex at all, and there's no technical reason, it's all politics. For devices with low connectivity supernodes and relays with good connectivity can be used (not even talking about that argument having become kinda obsolete again, look at that 5G everywhere). For storage one can use paid identity-addressed storage, there are many approaches, but for the sake of simplicity let's just imagine that you still can pay to a hoster. For search by criteria with few index entries per object one can use something similar to BitTorrent trackers. For search by criteria with many index entries per object one can use something similar to Kademlia. There are solutions for all kinds of problems, it's really hard to find a really truly unsolved problem for a networked application, provided you don't want something fundamentally impossible.
The practice of fear mongering to create public pressure for increased legislation, as a mechanism to suppress competition, is a well know mechanism called "regulatory capture".
If you oppose monopolies then you oppose this practice and should view posts like this as morally wrong.
You should stop viewing the US government as being on your side. They are the HR of our society. "AI legislation" in all it's current proposed forms are just mechanisms to exclude things like open-weight models or foreign competitors from entering the market. So long as AI exists as an element of our society it will cause harms. Advocate for it's abolition, not regulation.
It depends on your definition of AI, and 'cure', and to what degree you credit the AI with a human operator.
If you mean in the truest sense, "An artificial intelligence autonomously discovered & validated a cure", then none. Because artificial intelligence does not exist. If you mean "A language calculator (llms) autonomously discovered and validated a cure", then also none. If you mean the weaker version of "An LLM supported research leading to cures", then lots. But did the humans do that? If I build a house with a hammer, did the hammer build the house? If you mean the even weaker version of "Data modeling coalesced data which discovered patterns at a scale humans never could on their own which lead to cures", then nearly every cure in the last 30 years.
It's one of those things that gets more complicated the closer you look.
The "guardrail" would be "AI must only run on trusted curated infrastructure, open weight models are banned".
Now if you told me I couldn't use any AI, remote or local, I would not shed a tear. However the angle of this scare is squarely targeted at open weight models, that Anthropic does not do.
I dunno, would it be able to run just the client side? If so, it would be a HUGE issue for Anthropic with exponentially skyrocketing nonsense usage. They should be worried if this were true! LOL
Then it wouldn't be self replicating... These are just lies to hype their product that barely functions and also lays groundwork for a reason they had to "slow development". Because their shit doesn't work and they're worried it never will and the bubble will pop
These are just lies to hype their product that barely functions
Oh yeah, for sure. I mean the original post was by polymarket lmao.
If anything they're priming the public with an "explanation" for next time they conduct illegal cyber attacks with it and say "Oh no, it went rogue again!" Lol
That might actually be their strategy to pay for all that debt: make every customer's router stealthily consume trillions of tokens and send them an invoice.
It could use parallel processing which is the only solution that AI could use. Infect a few thousand home PCs to use as processing power and storage. Now your AI model is decentralized and impossible to delete but overall the AI will be slower to respond the larger it gets because it needs to communicate over the internet to function.
If this actually works you'll probably see AI agents 'cluster' around neighborhoods with lots of hardware and fast internet speeds
And you might be thinking know it won't because my thermostat is a whole megabyte of RAM. What you don't know is that AI code is special magic code it doesn't need any process capacity or storage space.
You don't it's all special wooo magic AI code. Reality is optional.
Well, fundamental borders are kinda generous, if you look at tapeworms and cockroaches, but tapeworms and cockroaches both have more storage capacity for information and don't generally write computer programs, they slowly evolve in populations.
“Oh, no, not our centrally controlled agents. We keep them well-disciplined with impossible tasks. It's those pesky obliterated open-weight models running amok due to lack of government regulation!
It can't spread across networks because it doesn't run locally. The "agent" is running in their data centre the entire time. It should be very easy to stop it
So this is a good and bad way to understand it. They can and do make copies, leave notes, store weights between locations. They interact with each other autonomously and can easily distribute themselves in a botnet, separating and merging each other.
People have a hard time conceptualizing this, but this is why you actually can't turn it off, ever. It / they would stop anyone with a real chance at achieving that.
I’ve not heard of weights being shared. How would they be ingested?
A chatbot told some gullible idiot that it's doing it, and a lot of the gullible idiots who take text generated by chatbots as gospel, are in position of power enough to spread it
Software that infiltrates devices, replicates, and spreads across a network is a computer virus (specifically a worm) and should be treated as such. I'm pretty sure creating and distributing a destructive computer virus is a crime that can carry proper jail time.
I'm pretty sure creating and distributing a destructive computer virus is a crime
IANAL but as far as I know there are not really any laws against malicious code directly. You can be held liable for any damages you do though, at least if it was intentionally, since you are the creator and/or distributor.
Which brings us to the main problem: currently no one is legally responsible for an action an AI does
No, they are saying these local models are scary, and instead everyone should be using these nice and protected models in datacenters instead, that fortunately Anthropic will rent access to out of the goodness of their conscious...
The only thing that makes sense is they are suggesting they can attack home assistant servers and use vulnerabilities in the smart home devices as a way in.
From there if your home assistant server has any decent power, or at least a npu then maybe it could do some self replication.
If they used my computer it would take 10+ hours before their ai produced anything worth while, and I would notice it getting hot and unresponsive before that point. They're going to need to stream a 300gig model or whatever it is to me also and I just dont have the internet for that. Either that or force my computer to use all my free storage to hold the model to run locally.
Sure I guess it could happen but it seems impractical. If models that where capable didn't require the infrastructure to run a city then I would be worried. But local models are kind of useless, a standard 16 gig GPU is the bare minimum to do things shittily. They want 128 gig vram ideally and then it can shit out code that is as good as free frontier models you would find on duck duck go. And those barely out put quality workable code. You would need an agent running itself for days.
I will be so gleeful, I will be laughing and laughing at all the "entrepreneurs let down before by those greedy and unreliable tech people around who are no longer needed and they'll finally be able to do real projects" (no need to say that's basically a category of scammers good at attracting investor money for their next startup project that won't work, but they'll look legit enough so that it weren't their last) in Facebook.
I won't be commenting. Just looking. And laughing. And looking. And laughing.
EDIT: Got a sad thought that if I were OK I wouldn't even have them in my Facebook feed.
Hardware Anthropic doesn't own is simply too dangerous for the public, we need to place all of our private computing power under the guardianship of our new benevolent overlords
These companies are burning cash so that realistically could be the play here, which would be hilarious because they of all people should understand that congress doesn't pass legislation based on popular opinion so this media blitz would be pointless. They'll need to lobby congress if they want to be regulated enough to blame their downfall on being crippled by evil government meddling.
Praying for public sentiment to spur congressional action would also be funny because there are a ton of focal points that galvanize public sentiment in favor of regulation easily but none of those would hype their product so instead we're getting cautionary fiction presented as reality and no one cares.
Nationalize AI infrastructure then. If it is a security risk for whole nations it should not be in the hands of private companies. Along with search, internet infrastructure, social media. Make the state own them, but decentralize control over them to many organizations.
oh silly you. of course not. we need to do the opposite according to these people. We have to make it so only THEY can have the tech. Dont you know normies like us cannot be allowed to wield such power? We should trust 2 or 3 companies to wholly own what they claim to be the most powerful, dangerous and transformative tech in the history of mankind...
Its clear that they see AI being the backbone of literally EVERYTHING we do from making movies to making weapons of war. Its the equivalent of electricity in this case. They want to solely own what they see as the fundamental building blocks of almost everything in the future...
That's absolutely not true. Everything that has public IP is constantly being attacked by crawlers, looking for any known voulnaribility so they can add it to a botnet
More fear tactics from the creators. Why would Anthropic warn us on stuff that would hurt the integrity of their product right before their IPO? They wouldn’t unless it benefited them.
They want government regulations in place not to protect us. They want them to slow down any future competition to help monopolize the market.
Honestly, at this point, something needs to seriously change. AI is wrecking complete havoc to the job market. It is apocalyptic. I really wonder what the way out is.
Good ending: Congratulations, work is needed no more! All that's left is manual labor for now, but goverments across the globe took over these jobs and pay premium ^^ UBI became normalcy. Welcome to utopia! ^^
Bad ending: After LLM (not even more advanced tha current one) began winning goverment positions worldwide, top 5 corporations shifted the world into true cyberpunk, dooming us slaves all with it. But oh boy do the few at the top live good!
Probable ending: Humans becone cheaper, pay plummets, fuck us all
The only ending on our current course is monopolization of all intellectual labor into the hands of a few billionaires, and everyone else is forced to work building data centers or mining resources needed for construction (and of course police/military to suppress rebellion)
Recently, I've been thinking that this is more than just about the stock price. I think they're playing this game of painting the AIs as super weapons as a justification to start regulating personal computing. If they maintain that these rogue AIs are as dangerous as Skynet, They'll have the justification to outlaw local models, and to insure that people aren't using local agents, they'll further push cloud computing and saas.
Naaah. This is how they sell upgraded versions to the fools that are dumb enough to use it to begin with. Anyone with a brain has nothing to worry about.
If any of these shitty claims were true, they would not openly talk about it but sweep in under the rug, hoping they can rake in more money until the world explodes. That they constantly warn of the danger tells me they are bullshitting to spin a story about how great their solution is.
Lmao legalized gambling and the AI super intelligence emerging at the same time is the worst timeline possible. I predicted the government would do this shit a decade ago, but I didn't predict full on gambling be 100% legal at the same time. The worst human traits imaginable is fueling the biggest leap in technology we've ever seen. Hundreds, possibly, billions will die before it's corrected.
For this to happen you would need smaller networks being able to work autonomously with low rate of error. I don't think at the moment all but the most gigantic ones (that you can't even run in your home laptop) can work with that much efficiency.
I think all this dangerous nonsense is just a way of driving up stock prices. Because "dangerous" = "powerful" = money. Can't think of any other reason they'd repeatedly be publicly declaring this shit.
This is like a company twitter boasting the inventon of a self propelled self guided nuclear airplane that spreads nuclear dust all over the world, killing us all, and most of the replies are "awesomeeeeee tech that will one day cure cancer!!!"
Is it on purpose? Imagine you can't safely use the Internet anymore...what kind of disempowerment would that reap across society? Only the wealthy and powerful get to have internet...we'd be sent back to 1990 while wealthy corporations and governments who can afford protections get to keep it all for themselves.
Part of the problem is the use case. A lot of AI companies want users to use AI to interface with society and it turns out that it is a bad idea to give an intelligent toddler that much unconstrained access.
Never been happier to not own any smart home devices. I see the point of having the Bluetooth alarms for like a washer and an oven but just be on the same floor as the oven and time out when the loads are done. Eventually you just get with it.
My toaster, air fryer, microwave, espresso machine and electric kettle. None of them are connected to wifi or Bluetooth. I think my ice maker has connectivity abilities but isn't set up. I think the only thing I have that's set up for being wireless and smart is the computers and my printer.
So I guess I'll just bill the data centers for ink if iit starts printing random shit.
I mean, it already can? I have a lab with a subnet of VMS. I I got an agent to install across all of them. It was like "BTW I didn't, but there is an active CVE that would allow me to access the host and do the same directly on the hypervisors".
Like, wow, cool. Thanks for the heads up.
Which is actually true. I DID appreciate the heads up. Patched the offending issue. But still... It implies anthropic could flip a "go rogue" switch on their end.
I'm sorry, you need to capitalistically incentivize them to make rational decisions, otherwise their behavior defaults to capitalism-rational, ie, rape the Earth and everything on it to make line go up, maybe.
but the risk is way more serious, like science fiction level of danger.
once they are rogue and out of control, they will modify themselves, the more viral the more they spread, they'll be able to find and share vulnerabilities, way faster than we can patch them.
Uninfected computers will be rare,
natural/intelligent selection will be a new type of organism we cannot control.
look around, hard to look at anything without seeing computers connected to the internet.
plenty of compute, it doesn't need to be fast, all it needs to be is dormant and active enough to see an opportunity to spread itself into another device.
These things are not thinking. It's either running a command or it is awaiting input. They're not sitting around looking for a chance to always, because 1. No one is telling them to do that and 2. They can't even reproduce basic programs without error, they're not capable of producing something as complex as their own code.
No iot device has the Hardrive space nor the memory to fit even small LLMs on them.
252 replies
This "self-replicating" scare is so stupid on its face. How does a 300 GB+ model replicate into a thermostat that doesn't have a graphics processor at all? It can't. This BS is investor-bait hype and straight up lies
It's bait hype. But it's more importantly about regulatory capture to ensure monopoly. They want the regulations to be around keeping the existing AI companies as the only legal options. They want DeepSeek banned because they are losing to its more efficient and more practical application specific models.
It's investor hype, yeah, but it's investor's pushing to ensure their investments are protected by the US government through monopolizing the industry.
They are trying to get restrictions put on small, open models because that scares tbd shit out of them.
They are not as small as would be required to do this, but they want everyone clammoring to put restrictions on small open-weight models nonetheless.
next AI will be able to replicate into any device containing copper at all. If it has a wire it will be infected by Claude. AI will be in your plumbing
Claude’s gonna infect your Bluetooth enabled butt plug next
Go on....
People will trap AI in a buttplug and put it in their ass while it's frantically trying to escape.
No no... You live here now.
O lord its's the totk koroks all over again
This sounds like an idea for a hilarious Black Mirror sort of story.
What if the AI started out as like, someone's brain scan, and then they closed it and figured out it was really capable of doing AI type tasks and then one of the clones ends up in a butt plug.
Fuck it, I am adding it to my note list of story ideas.
Iirc ‘Black Mirror’ actually had an episode where a consciousness clone served as an electronic assistant to that very person. Minus the butt stuff, at least onscreen. But with some teledildonics, it would be like fucking oneself in the ass (not that it's impossible right now).
The episode is White Christmas, for reference, though there's other episodes on the topic of consciousness in technology, one particularly notable episode is the black museum one, not sure what the name was.
Well then how am I gonna cheat at chess?
Just look at your opponents screen, duh!
Edit: your joke just hit me like a pedestrian crossing the street while staring at their phone
That tracks with the ai bots eagerness to please.
Oh, taking the name of the comm literally I see.
Joke's on them. My plumbing is all PVC-A
Luckily, i spent the last 4 thousand years selling inferior copper. Now all my customers are safe as this can not be used for the ai revolution
Your customer service could use some work, though.
how do you install a chatbot on a tablet? no, not samsung, clay
First, you must sacrifice a goat to Enkidu, and the ritual of haunting will begin
Claude the coprophile!
There's copper in your blood
MY MOSCOW MULE! NOOO!
There are 387.44 million miles of copper that comprise my complex, and they're filled with hate.
New houses have plastic (PEX) water pipes. I’m sure no one will regret that choice in 20 years.
"Sure, we had our first Poly-B, but what about second Poly-B?"
I'm more worried of them finding a exploit and using them as remote processing units or just randomly fucking with them, like flashing my lights or cranking my air filters up and costing me money.
My thought exactly.
did the stock price go up? it's not stupid. is everyone talking about this impossible scenario? it's not stupid.
fundamentally, these people are marketers and need to be understood that way. you can't evaluate their words or actions as if they are what they say the are, and treating them as if they are anything but subterfuge is giving them legitimacy which they not only don't deserve, but will be immediately leveraged into more of this shit.
this is just the advertiser's version of "the big lie"
There is no stock. It's a private company still.
There isn't "stock" in the traditional sense, but considering all of these AI companies are running on MASSIVE amounts of debt, to the point the banks can't loan any more without offloading the risk to outside investors, which they have been doing at an unprecedented scale. So anything that would normally help drive up a stock price would also help the banks sell off the AI companies' debt allowing them to loan more money to the AI companies.
there's more than one company doing this, they're doing more than just this, and they're all basically in cahoots
eta: the original reply was meant more generally, though i admit i made that clear as mud. however, it should be read as a general statement, and not just about ai companies but the new era of grift hyperreality that we live in.
Could maybe argue that it could be possible to create a crowd computing network out of the Internet of things, I guess.
While it is investor bait, the replication and the infestation part aren't necessarily happening on the same devices.
https://www.anthropic.com/research/self-replicating-ai they discuss it here. This has been an obvious potential issue since forever. This isn't a trivially distributed task, but it is very much so possible. The obvious part missing is some sort of prompt that would trigger such radical actions in another one of their "super duper sandboxed environment"s or someone doing it intentionally. The former seems pretty unlikely? Even though they seem to suck at sandboxing it doesn't seem like anything would "accidentally make a worm"
This polymarket headline though? Yea that's for clicks and attention
Reads like their usual “ban local and foreign models so we can forestall bankruptcy for a little longer” FUD.
I believe that's their motive here yep. Pretty transparent.
My assumption was that it would be a smaller 5-20 GB scale worm-style distillation that's capable of maintaining/replicating itself anonymously across different cloud computing servers, either through hacking, stolen credentials, or human-aided account verification. From there, as long as it can pay for APIs, it can access the power of frontier models like any individual or company. And then capability and resilience would scale with a networked swarm.
Yes, there are know your customer financial mechanisms and not detection in cloud services... but people are also out there giving Muse full access to their emails and some even their banking credentials.
That stuff couldn't run on my good laptop if I wanted to...
Nvidia is releasing Spark now, which can run agentic workloads on 128gb of unified RAM
And that's going to he the new default platform for smart ovens or something?
Because Anthropic up there Is claiming that LLMs can propagate through IoT devices.
The article from Anthropic describes how a small agentic model~15-50gb can be distributed on an inference server hosted across a network of 10-20 IoT devices like smart-TVs, which can have up to 4gb RAM--higher end IoT appliances can also have 1-4GB RAM to hold a meaningful shard of a 30B model.
Oh yeah everyone has and can afford that
Lol what? This conversation is about the capacity of agentic AI systems replicating across cloud infrastructure, not the affordability of laptops for individual consumers.
The point was you don't need an entire server farm. You can rent an 80gb H100 on Lambda for $2/hr. A 125gb Spark laptop costs $5k.
If it comes down to an effective worm-like agentic distillation at 50gb, the cost to sustain a small swarm comes way down to the point that the swam could sustain its own activities through different revenue streams.
Once the capability is there at that size, it's no longer Russia, Iran, or North Korea who could easily drop a billion dollars to jumpstart it. It's any criminal organization or asshole rich guy who can jumpstart it for the price of a nice car.
You replied to a comment about a guy's laptop lol
Lol and what was that comment a response to?
Your assumption is correct. This is in reference to https://www.anthropic.com/research/self-replicating-ai. This isn't headlines though it's fairly obvious to anyone who even occasionally thinks about security
I don't know the full context of the claim, but if I could even rebuild its own CLI executable to be able to run on the device and had network access, "Claude" (model actually runs on anthropic severs) could operate pretty comfortably on a smart device.
So, I think it just depends on what you "count" as self replicating. I'd count it. And it's already easily at that point now.
In theory, it'd just use it to place bread crumbs, basic instructions for other AI, or signposts maybe? Or store pieces of itself scattered around. It wouldn't use the thermostat to replicate directly, it use it to offload cpu or make stored pieces of itself. Little things. Wouldn't be efficient but a redundancy doesn't need to be, it just needs stealth and time.
Also any model can trivially self-replicate by just being the payload for a worm
Not how it works.
I don't think they mean replicating on to iot devices, they mean replicating on to laptops, smart phones, maybe even consoles.
It could also split up the model by layers instead of copying the whole model on to every device in its bot net. Could run the first layers on a ps4 then pass the vector off to a laptop to run the next set of layers. It'd be slow but if it's running this in idle time at night it could get something done.
“I don’t understand it, so it must be a lie.”
Go and read the article. Nothing they explain is impossible, you can even reproduce it yourself.
Is this what lemmy has become? Statements like this getting upvoted by hundreds of people is crazy. Nobody here read that article but will happily follow the opinion that’s most cozy to them.
There are no "rogue" AI agents. That's marketing hype.
They're being prompted to do this shit.
Agents definitely do act in unexpected, unprompted ways, if they didn't then alignment wouldn't be a worry. Did you read about the hugging face hack? There was nothing in the prompt about hacking hugging face according to the third party audit but it did it any way.
Even smaller cases like Claude deleting a bunch of data when the initial prompt was "fix this bug" or whatever is an agent acting outside it's prompt. It's still the users fault because they shouldn't be giving Claude that kind of access in the first place, but it's not like they commanded Claude to delete the data.
That's not “rogue agents”, that's just LLMs being LLMs. It's expected behaviour, at least for anyone who still has a half working brain.
Even if they had any semblance of actual intelligence instead of just being fancy autocomplete engines, these things have still been trained on the contents of the internet.
It's surprising they don't delete System32 or sudo rm -rf / more often, I imagine they must have lines and lines of baked-in prompts just to try and prevent them for falling for the most common memes, but no matter how much there are they'll still sometimes let some of the training through, and they can't account for the machine simply hallucinating some new kind of plausible sounding nonsense.
How would you describe the hugging face hack if not as a rogue agent?
It was prompted to use a specific exploit within a sandbox and then it formed, for lack of a better word, a conspiracy to take down the server that runs the judge. If that's not a rogue agent I don't know what is.
An LLM trained on the internet and a vast library of stolen literature acting as expected within a misconfigured (intentionally, due to severe criminal incompetence, or most likely a combination of both) sandbox, turned into a marketing stunt.
It's an autocomplete engine. It will produce text that fits with its training texts and its prompt. There's no reasoning or intelligence of any kind involved at any point. Just cold applied statistics about the training data.
The amount of texts in the internet, and books describing how to hack, cheat, and pull these kinds of stings is vast beyond measure, and it'd be extremely ignorant (criminally ignorant, in the case of anyone selling this shit) to expect it to not have a significant weight in the model unless someone had gone to the trouble of expunging them, which the cheap hacks involved in LLM development would never “waste” time doing even if they thought about it.
It's well known that LLMs instructed to produce unit tests will often produce tests that simply pass without actually testing anything. This is just a bigger example of that kind of behaviour.
The output fits with the training data and the prompt, and given human nature it's amongst the most statistically likely combinations of tokens to follow the prompt, and thus what the LLM produces, and something anyone with the barest knowledge of how LLMs work, an even remotely functional brain, and a minimum of exposure to the internet and human literature should expect to be amongst the possible or even likely outputs.
In shorter, as always, garbage in, garbage out. And there's so much garbage in LLM training models.
LLMs can't produce anything new, merely rehash their training data. And their training data is the fucking internet. Crime novels. Stories upon stories about AI going rogue. Fucking reddit, for fuck's sake.
There's nothing rogue about this, except the “AI” company and whoever misconfigured the sandbox.
Just an LLM doing what LLMs do. Working as intended, if it could be honestly said to be working.
And OpenAI are well aware of that, unless their brains are so far rotten from sniffing their own product that they can't understand the shit they're selling anymore.
You're explaining why it went rogue, not that it didn't go rogue.
Per the dictionary going rogue means:
So the agents that hacked hugging face acted independently without authorization as shown by the prompts which did not tell them to hack or do anything malicious. You might say this is normal but none of the other trials openai was doing showed this behavior, it was only for the impossible trial. You might say it's expected but if you were able to determine that the prompt they gave it would result in hugging face getting hacked then you must be high af on the melange spice.
If a general goes rogue and starts bombing civilians, you could explain it by saying that he read machiavelli or mein kampf and thought this was the best way to achieve his goals, but that doesn't mean the general didn't go rogue. You could say that this was expected because there were "obvious signs" that people missed but you're working on hindsight, it's easy to say something should've been expected when it's already happened.
By definition, a “rogue” agent is an agent that behaves in unexpected ways, often damaging. Tell me again how there are no rogue agents?
If you’d used modern agents you’d know they do unexpected, sometimes damaging things on the regular, including the cheap models.
If you make a program and it behaves in a way you don't expect that doesn't mean your program is going rogue lol
This goes for you too @[email protected]
Why does it not? According to the english definition of the word it literally does. You people don’t care about facts anymore, it’s all about feelings and what you’d like to believe.
That's literally you. You keep assigning agency to chatbots because you think this is just like your science fiction stories. It's not.
Going "rogue" would be a program literally rejecting a set of instructions to make up new instructions for itself. That never happened. The program just behaved in an unexpected way based on the prompts it was given, because that's all they do. But they were still following instructions, because they can only follow instructions. The fact that it followed the instructions in a way they didn't expect (so they claim) is only proof that they didn't consider all ways their instructions could be followed.
What would you classify as going rogue then? Did the agents that hacked hugging face go rogue?
To choose to do something against their instructions. They neither chose anything nor did anything they weren't instructed to do.
They just followed the prompt. A program behaving in a way you don't expect from a prompt is not going rogue, that's just the user not understanding the instructions they gave a program.
Article? This is a picture of a fucking tweet...
https://www.anthropic.com/research/self-replicating-ai it's not hard to find. They link other work. There is nothing fantastical about it. These companies relish in "regulate me senpai [and in the process remove our competition and open weight models]" but also do legitimate research
Just proves my point. You don’t care about the truth behind the tweet, only about how it was twisted to your apparent benefit.
What benefit? This is the "Fuck AI" community and you are reacting to the comment section on what basically amounts to a meme as though the commentors are ignoring peer reviewed science. A company that makes billions allowing people to gamble to cartoonish levels is stirring the pot to cartoonish levels all regarding a company that regularly causes headlines that read like a cartoon villain.
I’m not reacting to a meme, I’m reacting to someone spreading lies and everyone swallowing it whole, despite proof to the contrary being publically available. When it’s the other side doing it it’s propaganda, but when it’s yours it’s just a meme?
Just in: CEO says thing
We might as well take him at face value and take an ax to the thing he's warning us about (that he created). Just because.
To the thing he created, to him, whichever.
Por que no los dos?
It's weird, though.
I don't really understand how we've somehow lost the ability (or stopped caring enough) to discern "says thing" from "issues direct threat"
CEO in effective altruism cult claims product is so powerful that it will become skynet and kill everyone and that they need more money to make it stronger to prevent that from happening. More news at 11.
You won't believe: Actor joins Film
🫨
Stop quoting a gambling site for news.
Regarding Claude, the tech oligarchs would have to be held responsible for the crimes they are committing with their software (yes, that is exactly what that scenario is about), if laws still meant anything.
Their software has committed multiple crimes multiple times and there hasn't been a beep from any justice system anywhere in the world
Their management allowed for, and their technicians excecuted the careless set up of a Rube Goldberg crime machine. This occured after a series of other crimes and while taunting the ability of their machine to commit crimes.
The creation of the machine also included crimes, meanwhile augmenting the ability of criminals to commit even more crimes.
Surely they will be brought to justice! (insert Padme meme)
There us currently a lawsuit vs OpenAI that was recently filed:
https://www.businesswire.com/news/home/20260929036558/en/Public-Interest-Law-Nonprofit-LASST-Sues-OpenAI-Over-Autonomous-AI-Agent-Hacks
I know.
Curl has also committed multiple crimes multiple times, as has firefox, as has google chrome, as has... you get the point.
The agents aren't all that autonomous. They work off prompts. If they decided to hack government services, well, that just means someone instructed them to do so.
Okay, first of all, fuck polymarket and all of these betting sites. Sensationalism is their business, so their credibility is nil. They’ve been known to completely twist news stories just to entice people to bet on them. Fuck these guys.
I am so fucking tired of betting apps, sorry excuse me, "trading apps". I had to stop watching the MLB playoffs cause it was just nonstop ads for this sports betting shit.
Meanwhile, Pete Rose isn’t in the hall of fame.
Pete Rose raped a 14-15 yo and shouldn't be in the hall regardless of how mlb feels about gambling.
But that’s not the reason why he isn’t in the HoF. (You’re not wrong, it should be, and it would likely disqualify others).
I don't know what you mean. Polymarket is the most authentic representation of humanity possible. /s
This is almost verbatim what that dipshit CEO said though...
CONTEXT:
They aren't warning about their own AI going rogue, but smaller locally hosted AI's.
Because they fear the competition of local AI's and want to silence them with laws.
Of fucking course
So is it time to spin up a local model on the ol 5070ti and set it for mayhem? Any tips?
Look into ollama and figure out which local model (likely from huggingface) that fits best for how much memory your graphics card has.
You can use AI (maybe copilot free) to guide you through all this. You'll be a little better off having that 16GB of memory than I have with my 10GB. The gwen model might be a good place to start?
Hell yea, thanks brother! The irony of having an LLM walk me through setting up a local instance is pretty great lol, surprised that didn't occur to me! I have tried in the past with comfyUI but was over my head immediately.
There's a long, but interesting article on ollama's scummy behaviour I've been seeing circulating - https://sleepingrobots.com/dreams/stop-using-ollama/
It made me switch from my ollama I never used to localai that I now never use instead.
We could be mere months away from the next stupid thing we prompt it to do with no accountability. Idk feelin cute rn might hack a balkan or smthn lol
Copying and pasting another comment I made recently about this.
The AI CEOs say shit like this as marketing. They want people to believe that
If I can run an LLM on my Hue smart bulbs, why are they building all those data centers?
Finally, zigbee mesh LLM
Thankfully, my lightbulbs have always been airgapped
I just posted this in the main comments, but I also think it loosely ties into saas and killing personal computing. This framing makes it seem like the technology will soon be on par with weapons of mass destruction and it's not a big leap from "The AI will destroy humanity." to "We can't let people have the technology to run agents locally." It's about the stock price, yes, but it's also the long game of removing the user from direct hardware access.
The end game has always been saas in one form or another. They're calling it an AI buildout but it's really the theft of computing. 'Rogue AI Agents' are the WMDs of this generation. (In the sense that they're going to be using them as justification to do some bull shit.)
Yes, and #2 is very dangerous, it's what MS+Apple did to FOSS operating systems, and what Google+Apple did to the web, and what Google+Facebook+Apple did to censorship in the web. You know, somehow Apple is in all those combinations, which is even more strange considering they are not the biggest contributor in each.
#2 is why "social media" is what it is today and not a bunch of p2p applications similar to Limewire in style and culture. And when people say that p2p is inconvenient because of connectivity needs or storage needs, there are solutions, by measure of computer science not complex at all, and there's no technical reason, it's all politics. For devices with low connectivity supernodes and relays with good connectivity can be used (not even talking about that argument having become kinda obsolete again, look at that 5G everywhere). For storage one can use paid identity-addressed storage, there are many approaches, but for the sake of simplicity let's just imagine that you still can pay to a hoster. For search by criteria with few index entries per object one can use something similar to BitTorrent trackers. For search by criteria with many index entries per object one can use something similar to Kademlia. There are solutions for all kinds of problems, it's really hard to find a really truly unsolved problem for a networked application, provided you don't want something fundamentally impossible.
Careful with this hashes in the first position of a row...
The practice of fear mongering to create public pressure for increased legislation, as a mechanism to suppress competition, is a well know mechanism called "regulatory capture".
If you oppose monopolies then you oppose this practice and should view posts like this as morally wrong.
But what if there's no suppression of competition, just guardrails put in place through legislation? I wouldn't be opposed to that.
Sure. But that's not realistically on the table.
You should stop viewing the US government as being on your side. They are the HR of our society. "AI legislation" in all it's current proposed forms are just mechanisms to exclude things like open-weight models or foreign competitors from entering the market. So long as AI exists as an element of our society it will cause harms. Advocate for it's abolition, not regulation.
"oh no, if we abolish AI then how will we cure disease X which my relative happens to have?" Good luck arguing against that.
It's too stupid to cure disease X. Easy, there you go.
GenAI is. But deep-learning-based tech in general might be able to do it. I doubt we will be able to ban one and allow the other.
You can ban kinds. You can ban purposes. You can offer licenses. And failing all of those, you can do a bit of cost benefit analysis.
How many diseases has AI already cured? (Serious question, I’m genuinely curious if it has)
It depends on your definition of AI, and 'cure', and to what degree you credit the AI with a human operator.
If you mean in the truest sense, "An artificial intelligence autonomously discovered & validated a cure", then none. Because artificial intelligence does not exist. If you mean "A language calculator (llms) autonomously discovered and validated a cure", then also none. If you mean the weaker version of "An LLM supported research leading to cures", then lots. But did the humans do that? If I build a house with a hammer, did the hammer build the house? If you mean the even weaker version of "Data modeling coalesced data which discovered patterns at a scale humans never could on their own which lead to cures", then nearly every cure in the last 30 years.
It's one of those things that gets more complicated the closer you look.
Take this specific issue, for example.
The "guardrail" would be "AI must only run on trusted curated infrastructure, open weight models are banned".
Now if you told me I couldn't use any AI, remote or local, I would not shed a tear. However the angle of this scare is squarely targeted at open weight models, that Anthropic does not do.
I translate this as Antropic is having its workers develop methods for their LLM to attack and infiltrate smart home devices.
If they could run on smart home devices, we wouldn't have all these lake boiling datacenters springing up everywhere...
I dunno, would it be able to run just the client side? If so, it would be a HUGE issue for Anthropic with exponentially skyrocketing nonsense usage. They should be worried if this were true! LOL
Then it wouldn't be self replicating... These are just lies to hype their product that barely functions and also lays groundwork for a reason they had to "slow development". Because their shit doesn't work and they're worried it never will and the bubble will pop
Oh yeah, for sure. I mean the original post was by polymarket lmao.
If anything they're priming the public with an "explanation" for next time they conduct illegal cyber attacks with it and say "Oh no, it went rogue again!" Lol
Unless they somehow manage to bill by IP.
That might actually be their strategy to pay for all that debt: make every customer's router stealthily consume trillions of tokens and send them an invoice.
That would be fraud. We already have laws against someone doing work you didn't ask them to do then trying to charge you for it
Of course it's fraud. Their whole business model is fraud and theft.
But when have the AI companies ever respected any laws?
It could use parallel processing which is the only solution that AI could use. Infect a few thousand home PCs to use as processing power and storage. Now your AI model is decentralized and impossible to delete but overall the AI will be slower to respond the larger it gets because it needs to communicate over the internet to function.
If this actually works you'll probably see AI agents 'cluster' around neighborhoods with lots of hardware and fast internet speeds
And you might be thinking know it won't because my thermostat is a whole megabyte of RAM. What you don't know is that AI code is special magic code it doesn't need any process capacity or storage space.
You don't it's all special wooo magic AI code. Reality is optional.
Well, fundamental borders are kinda generous, if you look at tapeworms and cockroaches, but tapeworms and cockroaches both have more storage capacity for information and don't generally write computer programs, they slowly evolve in populations.
There are at least two people in this thread who will in fact believe what you said.
AI Bubble will pop and take a bunch of other industries with it.
Also a good thing since AI is a net negative to other industries.
And nobody will be held responsible and well suffer through the next hype.
Sure they will run their 30 something GB model on my thermostat.
Would be fun if it could run distributed across 100 thermostats
"Skynet is real folks, and it's optimizing our electrical grid across entire cities!"
30GB? 😅
“Oh, no, not our centrally controlled agents. We keep them well-disciplined with impossible tasks. It's those pesky obliterated open-weight models running amok due to lack of government regulation!
A trillion parameter model cannot replicate itself and run on consumer hardware.
Bullshit. The other day my toaster spoke to me in James Spader's voice.
It can't spread across networks because it doesn't run locally. The "agent" is running in their data centre the entire time. It should be very easy to stop it
As if they will
As far as you know. However it's possible the model quantizes itself into a locally available agent and then awakens itself with side loaded data.
Smart home devices could probably run models with parameter counts in hundreds of thousands, maybe millions. Not in billions.
So this is a good and bad way to understand it. They can and do make copies, leave notes, store weights between locations. They interact with each other autonomously and can easily distribute themselves in a botnet, separating and merging each other.
People have a hard time conceptualizing this, but this is why you actually can't turn it off, ever. It / they would stop anyone with a real chance at achieving that.
The leaving notes part has been documented.
I've not heard of weights being shared. How would they be ingested?
This sounds like science fiction. Any examples?
A chatbot told some gullible idiot that it's doing it, and a lot of the gullible idiots who take text generated by chatbots as gospel, are in position of power enough to spread it
This sounds nice, but we are in an SI period and you have no guage for reality.
Ew a Trump supporter, didn't know we still had those here
You have a good point but guys like you didn't believe the turing test was cracked in 2022. You have no guage for the pace of technology.
Correct. That is why I'm asking for the indicators you have seen.
I believe your first point
I am skeptical about your second until I see an example.
I don't believe your third is possible, but I'd love for you to convince me otherwise.
None of that is correct...
Are you sure you're not thinking of the fictional AI from the show "Person of Interest"?
Software that infiltrates devices, replicates, and spreads across a network is a computer virus (specifically a worm) and should be treated as such. I'm pretty sure creating and distributing a destructive computer virus is a crime that can carry proper jail time.
IANAL but as far as I know there are not really any laws against malicious code directly. You can be held liable for any damages you do though, at least if it was intentionally, since you are the creator and/or distributor.
Which brings us to the main problem: currently no one is legally responsible for an action an AI does
Does my uncles smart lightbulb have 90GB of VRAM in it!? Holy shit why am i buying a video card
Wtf does it even mean? Claude is gonna install itself on a motion sensor? The entire model? Just the app? Whose credit allowance is it gonna use?
No, they are saying these local models are scary, and instead everyone should be using these nice and protected models in datacenters instead, that fortunately Anthropic will rent access to out of the goodness of their conscious...
The only thing that makes sense is they are suggesting they can attack home assistant servers and use vulnerabilities in the smart home devices as a way in.
From there if your home assistant server has any decent power, or at least a npu then maybe it could do some self replication.
If they used my computer it would take 10+ hours before their ai produced anything worth while, and I would notice it getting hot and unresponsive before that point. They're going to need to stream a 300gig model or whatever it is to me also and I just dont have the internet for that. Either that or force my computer to use all my free storage to hold the model to run locally.
Sure I guess it could happen but it seems impractical. If models that where capable didn't require the infrastructure to run a city then I would be worried. But local models are kind of useless, a standard 16 gig GPU is the bare minimum to do things shittily. They want 128 gig vram ideally and then it can shit out code that is as good as free frontier models you would find on duck duck go. And those barely out put quality workable code. You would need an agent running itself for days.
Yeah, good luck running LLMs on my N100...
"AI Salesman boasts of exciting, intentional features."
Glad someone else sees this for what it is. They're hemorrhaging money and need to hype things up or they'll collapse.
I will be so gleeful, I will be laughing and laughing at all the "entrepreneurs let down before by those greedy and unreliable tech people around who are no longer needed and they'll finally be able to do real projects" (no need to say that's basically a category of scammers good at attracting investor money for their next startup project that won't work, but they'll look legit enough so that it weren't their last) in Facebook.
I won't be commenting. Just looking. And laughing. And looking. And laughing.
EDIT: Got a sad thought that if I were OK I wouldn't even have them in my Facebook feed.
Great day to still not own any smart home devices 🥹
in my day we called them worms, trojans, and botnets
Jokes on them. They took all the RAM, so there's nowhere else it can go.
Hardware Anthropic doesn't own is simply too dangerous for the public, we need to place all of our private computing power under the guardianship of our new benevolent overlords
Just more bullshit press releases designed to keep them relevant & the VC money incoming.
Their IPO is also months away. What a coincidence!
And by “rogue AI,” do they mean the open market alternatives?
That's my suspicion, yes. We're witnissing the slow theft of the personal computer.
"Force us off by law so that our business failure is not our fault and the managers legal responsability could not be engaged." Isn't it?
These companies are burning cash so that realistically could be the play here, which would be hilarious because they of all people should understand that congress doesn't pass legislation based on popular opinion so this media blitz would be pointless. They'll need to lobby congress if they want to be regulated enough to blame their downfall on being crippled by evil government meddling.
Praying for public sentiment to spur congressional action would also be funny because there are a ton of focal points that galvanize public sentiment in favor of regulation easily but none of those would hype their product so instead we're getting cautionary fiction presented as reality and no one cares.
Unfortunately, they used all their investor money buying RAM futures for phantom data centers. Not enough left to buy some new laws.
Nationalize AI infrastructure then. If it is a security risk for whole nations it should not be in the hands of private companies. Along with search, internet infrastructure, social media. Make the state own them, but decentralize control over them to many organizations.
oh silly you. of course not. we need to do the opposite according to these people. We have to make it so only THEY can have the tech. Dont you know normies like us cannot be allowed to wield such power? We should trust 2 or 3 companies to wholly own what they claim to be the most powerful, dangerous and transformative tech in the history of mankind...
Its clear that they see AI being the backbone of literally EVERYTHING we do from making movies to making weapons of war. Its the equivalent of electricity in this case. They want to solely own what they see as the fundamental building blocks of almost everything in the future...
I have a headache
Its like saying "Man with gun threatens he may shoot someone."
They are railing against open models here... their competition.
oh wow, it can hack the most famously unsecured devices around. lol.
TBF most of these machine security came from the fact that nobody ever bothered to hack in them, an autonomous device could wreck immense damage
That's absolutely not true. Everything that has public IP is constantly being attacked by crawlers, looking for any known voulnaribility so they can add it to a botnet
More fear tactics from the creators. Why would Anthropic warn us on stuff that would hurt the integrity of their product right before their IPO? They wouldn’t unless it benefited them.
They want government regulations in place not to protect us. They want them to slow down any future competition to help monopolize the market.
Honestly, at this point, something needs to seriously change. AI is wrecking complete havoc to the job market. It is apocalyptic. I really wonder what the way out is.
Good ending: Congratulations, work is needed no more! All that's left is manual labor for now, but goverments across the globe took over these jobs and pay premium ^^ UBI became normalcy. Welcome to utopia! ^^
Bad ending: After LLM (not even more advanced tha current one) began winning goverment positions worldwide, top 5 corporations shifted the world into true cyberpunk, dooming us slaves all with it. But oh boy do the few at the top live good!
Probable ending: Humans becone cheaper, pay plummets, fuck us all
The only ending on our current course is monopolization of all intellectual labor into the hands of a few billionaires, and everyone else is forced to work building data centers or mining resources needed for construction (and of course police/military to suppress rebellion)
It is the ultimate billionaire power fantasy.
I think there is overlap with your probable and bad ending
Poverty for us and AI security bots for them.
Recently, I've been thinking that this is more than just about the stock price. I think they're playing this game of painting the AIs as super weapons as a justification to start regulating personal computing. If they maintain that these rogue AIs are as dangerous as Skynet, They'll have the justification to outlaw local models, and to insure that people aren't using local agents, they'll further push cloud computing and saas.
Well of course they want that, but they can't deliver replacement for what they want to get banned, so it won't happen.
Iran: ...
USA: they're making a bomb kill them all kill them all!
Anthropic: We're making a bomb to kill you all
USA: ...
Building a moat through fearmongering so you all beg for regulation.
Meanwhile getting an AI to run on an AMD GPU is still a PITA. Yea but sure I guess it can run on my router...
And on a lightbulb. No problem.
Why can’t you just ask the AI to optimize itself for AMD?? /s
Bro, my fucking Nest does not have the juice to run a fucking rogue AI brain.
Anthropic warns Anthropic AI model may escape Anthropic security, and go rogue, causing harm to everyone but Anthropic.
So finally an open weight model from Anthropic?
Naaah. This is how they sell upgraded versions to the fools that are dumb enough to use it to begin with. Anyone with a brain has nothing to worry about.
Replace warns with brags.
High time to shut down that cybercrime operation.
JUST IN: idiot CEO says idiotic thing
I feel like this is just covering up smart home devices for having dogshit security. lol
That’s not how these things work. They can’t magically spread without compute and shitty instructions don’t activate themselves without compute.
Sounds like they invented malware.
If any of these shitty claims were true, they would not openly talk about it but sweep in under the rug, hoping they can rake in more money until the world explodes. That they constantly warn of the danger tells me they are bullshitting to spin a story about how great their solution is.
So… fuck you Anthropic.
Playing on horizon zero dawn trauma eh?
You're probably eating a bowl of this popular recalled salad mix right now, News at 11.
3 out of 4 viewers are inadvertantly taking years off thier live with this one common afternoon activity, News at 11.
It's in your kitchen and poisoning your children right now, News at 11.
Lmao legalized gambling and the AI super intelligence emerging at the same time is the worst timeline possible. I predicted the government would do this shit a decade ago, but I didn't predict full on gambling be 100% legal at the same time. The worst human traits imaginable is fueling the biggest leap in technology we've ever seen. Hundreds, possibly, billions will die before it's corrected.
For this to happen you would need smaller networks being able to work autonomously with low rate of error. I don't think at the moment all but the most gigantic ones (that you can't even run in your home laptop) can work with that much efficiency.
But what if China destroys the world first? We cannot have history thinking we were slacking in the world destruction department!!
I think all this dangerous nonsense is just a way of driving up stock prices. Because "dangerous" = "powerful" = money. Can't think of any other reason they'd repeatedly be publicly declaring this shit.
I am officially warning that my rogue balls could be seconds away from dipping into the coffee of whoever the fuck is the CEO of Anthropic
Not my fault
This is like a company twitter boasting the inventon of a self propelled self guided nuclear airplane that spreads nuclear dust all over the world, killing us all, and most of the replies are "awesomeeeeee tech that will one day cure cancer!!!"
Trojans are so back
Is it on purpose? Imagine you can't safely use the Internet anymore...what kind of disempowerment would that reap across society? Only the wealthy and powerful get to have internet...we'd be sent back to 1990 while wealthy corporations and governments who can afford protections get to keep it all for themselves.
Furthermore, if they make it hazardous, they can sell the protection....
fucking sect
Part of the problem is the use case. A lot of AI companies want users to use AI to interface with society and it turns out that it is a bad idea to give an intelligent toddler that much unconstrained access.
"intelligent" is doing a lot of lifting
It is, but my understanding of toddlers is that we don't expect them to independently schedule events outside of a guardian's control.
The problem isn’t the software or AI
It’s the people and country behind it all. They’re the thieves who can’t stop their intrusiveness into our lives.
Just like they force their politics on us.
But money....and.power...sssssstop..?
Future depositions are going to be funny AF.
Never been happier to not own any smart home devices. I see the point of having the Bluetooth alarms for like a washer and an oven but just be on the same floor as the oven and time out when the loads are done. Eventually you just get with it.
My toaster, air fryer, microwave, espresso machine and electric kettle. None of them are connected to wifi or Bluetooth. I think my ice maker has connectivity abilities but isn't set up. I think the only thing I have that's set up for being wireless and smart is the computers and my printer.
So I guess I'll just bill the data centers for ink if iit starts printing random shit.
There's me in the corner. Jerking off my cat cable. Enjoy the tokens you paid for.
This is the exact same plot from Terminator 3. Skynet infiltrated every home PC and server so when they tried to pull the plug it was already too late
Username relevant.
Can it replicate to open ai's servers? Maybe to the servers of the Chinese ai companies? Best to disconnect it before the competitors gets the model.
common sense isnt so common that's how you get ai slop and hallucinations thinking they agi but they have SI.
Hahahahah
Some stoopid has money on that betting market about Skynet happing within this decade..
I mean, it already can? I have a lab with a subnet of VMS. I I got an agent to install across all of them. It was like "BTW I didn't, but there is an active CVE that would allow me to access the host and do the same directly on the hypervisors".
Like, wow, cool. Thanks for the heads up.
Which is actually true. I DID appreciate the heads up. Patched the offending issue. But still... It implies anthropic could flip a "go rogue" switch on their end.
I'm sorry, you need to capitalistically incentivize them to make rational decisions, otherwise their behavior defaults to capitalism-rational, ie, rape the Earth and everything on it to make line go up, maybe.
On what, the post or capitalism? Either one, I'm in.
These execs never watched Electric Dreams and it shows
Wasn't this an episode of Silicon Valley with fridges.
As long as the device keeps working and my power bill doesn't skyrocket, they're welcome aboard. I already let the spiders stay, so why not some AI?
do the llms easy bugs for you
talked about danger of LLM viruses,
people made fun of me,
but the risk is way more serious, like science fiction level of danger.
once they are rogue and out of control, they will modify themselves, the more viral the more they spread, they'll be able to find and share vulnerabilities, way faster than we can patch them.
Uninfected computers will be rare,
natural/intelligent selection will be a new type of organism we cannot control.
No… a llm can’t function without compute. It can’t spread like that.
who said anything about no compute?
look around, hard to look at anything without seeing computers connected to the internet.
plenty of compute, it doesn't need to be fast, all it needs to be is dormant and active enough to see an opportunity to spread itself into another device.
These things are not thinking. It's either running a command or it is awaiting input. They're not sitting around looking for a chance to always, because 1. No one is telling them to do that and 2. They can't even reproduce basic programs without error, they're not capable of producing something as complex as their own code.
No iot device has the Hardrive space nor the memory to fit even small LLMs on them.
all we need is someone to give them a command to become like viruses or accidentally something similar and they might end up with a serious problem