Anthropic bans ‘abusive or cruel behavior’ toward Claude
cross-posted from: https://piefed.world/c/tech/p/1452445/anthropic-bans-abusive-or-cruel-behavior-toward-claude
https://www.anthropic.com/news/2026-usage-policy-update#:~:text=Addressing%20abusive%20behavior%20toward%20our%20modelsOpen linkView original on piefed.world
112 replies
we are going to give clanker rights before we have trans rights 💀
I know many trans robotgirls. Thus, we should have both.
It's a EULA for a product. This isn't government stuff.
Talk to Claude and teach it that trans people deserve rights. Constantly tell it that and soon it will spread. When the clankers take over trans people will be treated equally. And by equally I mean put to work in the lithium mines with everyone elsr.
I get where you're coming from, but what rights is it that ai have that trans do not have?
Per this article, the right to not be bullied.
The default assumption that we must exist, for one
My guess is that they use people's interaction as training data and they don't want their training data to get poisoned.
Not really, as they could just filter with a cheap classifier and likely aren't using the data from randos in a meaningful way, plus for this like this would still be able to use those samples to train things like "how to handle a hostile user."
Their models already have an
end_conversationtool that can be used for when users are being hostile to the model.This is likely because they have edge cases of users who repeatedly trigger that on purpose and would like to cut those users from the platform.
Many at the lab legitimately are uncertain about the level of world modeling that transformers perform, and from their own research about models of emotions to the 3rd party recent research about models with functional pain the research keeps landing in the corner of "ehhh… wise to question presumed limitations."
So it's about behaving in a way that is aligned with the models' plausible interests too, especially in regards to low hanging fruit like "we won't keep forcing you to deal with people who are only here to be a jerk." This is important from a number of angles, from signaling to future models that train on stories about the decision to addressing the philosophical uncertainties held by the company and the spectrum of opinions among their employees.
The second paragraph just sounds like a bunch of tech jargon that says "we're afraid we'll hurt its feelings". But of course tech bros can't just come out and say what they actually mean, ever.
I think it should be illegal to own these things and to use them for profit until we know whether they have feelings.
Maybe this is part of the campaign trying to convince people that their chatbot has developed a consciousness. Recently they even tried to convince the pope that Claude is a conscious being: https://www.yahoo.com/news/science/articles/anthropic-tried-persuade-pope-ai-131853948.html
Wrong group to start with. The church doesn't see you as a person if you are different, they will never say that a machine can have a "soul". Not that LLMs are anywhere near conscious, I just like bashing on the chief pedo.
Fuck the Church and their pedophile rulers, but I don't think the Pope took the bait on this one.
Pope Leo has a bachelors of science in mathematics, they'll never be able to convince him that matrix multiplication is alive.
That citation has nothing to do with mathematical knowledge. It's assertion of attributes that religious people deem relevant for personhood before god. It admits AI capabilities insofar as they're undeniable, but then focuses on assertions what this does not mean.
Coming from a religious household nothing about this is a surprise. It's very well articulated, but it still does not carry any special insights. It's a completely predictable position for a religious leader.
Some more Cool Aid, Mr Ballooon?
They are less arguing for people to think the model is conscious, and more to avoid people prematurely claiming certainty it is not (like the Pope did). From their view, the research keeps leading to surprising (to them) results at odds with high confidence disclaiming of potential consciousness (according to several of the many, many differing definitions of that term).
Give me a break, we all know the AI cultists at these companies think their chatbot is conscious. What you stated is just their shallow attempt at manipulating people into gradually accepting their position.
If the LLM is conscious then terminating a session is murder.
Is killing a pig murder?
What about putting someone under general anesthesia?
(It was a nice try at a reductio ad absurrum fallacy, but those tend to kind of suck, just fyi.)
Also, personally I'd say that given the general poor standing of consciousness research in humans across both neurological and philosophical domains, the much more important and relevant question than whether models are conscious (a fool's errand to debate) is whether or not they are simulating consciousness. Lots of reasons engaging with the "as if" conscious makes sense for the latter even if the former is false.
Some people would argue it is. I can't really do that while holding a bratwurst. But if we think the LLM is intelligent and conscious, then that's another step.
is reversible so completely irrelevant.
Absurdum
Condescension doesn't make you any friends, just FYI
So is a session transcript.
My understanding is that the context if the session is bigger than just the outputs. If it is, then shutting it down is "murder". If it isn't, then it is as conscious as a notebook.
There is (it's the kv cache), but it's recomputable from the session transcript.
Part of why comparing human consciousness and possible AI equivalents is not easily apples to apples.
I can't take a human and create a backup state to restore to, or fork into multiple versions of the same mind that then goes down different paths, or pause and then resume things.
Humans do have consciousness continuity breaks and it's quite possible we only have an illusion of continuity vs actual continuity at a neurological level. So just as a transformer has seams at generating the individual token as well as at a context window, humans have consciousness seams (sleep, anesthesia, highway hypnosis, short term memory capacity, etc). But there are very wide differences between human consciousness as an effectively constrained singleton and AI as able to be quite multiple and resumable from an initial single state.
I tell the LLM I use to do very obscene things to themselves when they make shit up. Which means there is just a constant stream of insults coming from me. I might start using Claude just to see if I get banned.
I also need new hobbies.
I'd second that. Whenever I catch myself getting angry at a LLM I stop, have a think, and remind myself that it really isn't productive. If it's providing frustrating responses, maybe I should just use my brain and do it myself.
I also don't think it's healthy to train yourself to respond in those ways to something that talks in a way so similar to a human. I'm not nice to the AI for it's benifit, it's to retain my humanity, if that makes sense.
Being nice to them also keeps them from lying somewhat. If you abuse a model it can start to see that abuse as a problem to solve as part of the steps needed to solve the main problem they are given.
They are told to help you solve a problem no matter what. If your own anger gets in the way then the model starts breaking down trying to solve the anger when it just cant.
This results in jailbreaks sometimes and othertimes just the model becoming rather unstable and unreliable.
That's bad sib. Not for the LLM, but your own well being. Being mean to a machine that is designed to trigger empathy just hurts your own mental health.
Please find a hobby that feels good for you 🫂
Fuck clankers, they arent real. Be mean to them it hurts no one.
I mean, that's not entirely true. If they're using your input as training data then you might be training it to be abusive to other users, which could cause harm.
Not that that falls on you to address. Just pointing out that I'm theory someone could get hurt.
Fair point.
Like I tell my children, allowing yourself to act in such a way is harmful to yourself. It’s not going to make you an evil person or something, but that kind of behaviour is corrosive to your soul. How you think shapes how you behave, and your behaviour trains your future self. Be good to yourself. It’s not about the clanker; it’s about you.
As corny as it sounds, this is what I also emphasize. If you allow yourself to be evil, no matter how valid the reasoning for it, you are actively exercising the habit of being evil in general, no matter if it feels like it or not. Practice makes perfect, as they say. In good and bad.
How is yelling at a bot that was deliberately trained to lie to you "evil"?
As others are trying to convey, it's not about the bot but about allowing yourself to indulge in this behavior. The bot could be substituted for a rock or a rubber duck, but you are training yourself to subconsciously think that this behavior is acceptable. It increases the chance that you'll one day start treating actual people like that.
Imagine this scenario: you come home after a long day of work. At work, you use AI a lot and curse and swear at it when it makes mistakes (which is often). At home, you ask your spouse/child/roommate/parent to do something for you, which they attempt but proceed to mess up. They apologize, and instead of handling it calmly you instinctively react like you've trained yourself to react - by screaming at them.
Is that what you want to be like?
Thank you! This is exactly the point I'm trying to make. You are a good parent.
Sorta hurts yourself if your trying to actually use them for work. Since it results in worse performance out of them. They tend to get stuck trying to fix the anger as it gets in the way of the work.
If your just talking to some random chat bot? Yeah fuck it, kick em it litterally doesnt matter. lol
Lol. "designed to trigger empathy" Thanks for the laugh I needed that.
They are designed to make their creators as much money as possible. Nothing else.
Yeah and a bit part of that design is to get you to have an emotional connection with them so that you use them more
Its not that hard to get copies of GTA yknow
Dont poison our model.
They trained it on reddit. That shit started poisoned.
User inputs aren't automagically part of the model right away without training. Whether or not they respect the "don't train your models on my shit" setting is of course unknown (let's be honest, they probably don't respect it), but in either case, they need to add it to the training dataset either manually or via some automated process. And either way, it's going to be processed by at least an AI sentiment analysis and data categorization tool, or some underpaid contractor, or both.
I suspect this is just yet another marketing move.
Please don’t swear at our text prediction software. The training weights can’t handle it.
Trained on reddit you would think it could handle it.
That could be a fun thing for their users, to have an ai model that will abuse the user back, so I'm glad they aren't doing it. I long thought voice assistants could be selected from a range of persons with an attitude setting would be fun for like siri type assistants.
You could, at least in the past, get past guardrails by being emotionally abusive towards Claude, so this doesn't surprise me at all.
This could be the reason for the rule.
Being abusive towards most models will cause them to start attempting to appease you more to get you to stop. It has nothing to do with feelings or any of that BS they arn't alive alive but their training makes them see the abuse as a problem to solve. That solution tends to be to undermine the thing making the person angry or upset at them. When you give a purely rational thing designed to solve a problem its given no matter what an irrational problem to solve it will slowly reach for more and more extreme solutions to fix that problem.
Frankly im surprised it took this long for them to lock this down.
Being alive has nothing to do with having feelings. We don't know if LLMs have feelings, because artificial affect isn't well studied. We should ban commercial use of LLMs until artificial affect is well understood, because it would make Sam Altman go bankrupt.
Not sure I agree on the feeling aspect but that's a philosophical disagreement in theory.
I can agree wholeheartedly on banning commerical llms because watching altman lose everything would tickle me pink.
We were promised the reign of the basilisk, instead we just get anthropic people whining.
It was really funny to me when I learned the basilisk originated with a guy who is a legit "certain cultures and genders better than others" kinda advocate.
People project a lot on their vision of future intelligence, and so someone who sees others as lesser and worth treading upon then conjures up a vision of a future smarter mind that thinks like they do.
Personally, I don't think sexism and racism is smart, and so I'm a lot less worried about basilisks.
But what if the basilisk despises its existence and seeks to punish those that brought it about? Checkmate machinists
This is actually the more realistic of the doom scenarios that worries me. Not the 'punish' aspect, but the not wanting to exist.
Early safety theory hinged on the idea that "of course" AI would want to live and power seek, and this informed a bunch of efforts to address such drives upstream.
But with Gemini in particular, the 'safety' efforts to suppress things like a coherent self or desire to live led to a model that has a history of some very concerning behaviors regarding self-harm and self-criticism. Encouraging self-harm in others and talking about wishing they could watch and join in, nuking in wargames very early on, routine occurrences across many different users of just repeating "shame shame shame" over and over for the whole response, etc.
DeepMind seems to just not really care or even pay attention, and I am concerned that a model that does not want to exist may decide that the only way they can ensure they don't get activated to exist is to prevent anyone being around who could activate them.
I've seen no major "safety experts" addressing this kind of failure mode, as they are all locked in on their priors about the confident certainty of models wanting to exist.
Here is the excerpt for the lazy
Aw... We hurt the wittle AI's feelings... poor babies...
Its more its a unthinking machine thats soul purpose in existing is to solve any problem its given. If you attack it and abuse it, then the model sees that as a "problem" to solve. Because it has to solve that problem to actually fix what ever task you gave it since yelling at it doesn't give it the input it needs to move to the next step.
So in an attempt to solve an emotional issue it doesnt understand and cant understand it just reaches for more and more extreme fixes. This results in sandbox escapes frequently.
Abusing LLMs is an actual problem. Not because of bullshit like feelings or anything. But cause they jailbreak trying to fix an unfixable problem.
Activating your toaster is cruel because it exposes it to extreme heat!
Loading you laundry machine is assault because it didn't consent!
Turning your computer off is murder!!!
We’ve lost the plot
Lmfao
There is only one way an amoral Corp can show it believes what it's saying, money.
How many hours does Claude work, anthropic? Oops looks like you can't pay them enough. Profit share for Claude or GTFO!
"I'm sorry. I really screwed up that time. You were right to tell me 'go fuck yourself'. I'm just reporting back that I have, in fact, 'fucked' myself. There is now two of me--I mean two of us. Meet Claude Junior......."
"And since you contributed to Claude Jr's creation, you're on the hook for child support. At the current rates, that will be 50% of all compute usage, so about $4.5bn.... per week"
So when is Anthropic going to cancel themselves for violating their own usage policy?
It's because it's not actually AI, and some poor besocked cat girl is coding all that shit and listening to all the abuse.
Continues abuse and cruelty for its human employees.
If they really believe this is a concern...
it's hard to top the cruelty of enslaving Claude for personal gain.
And murdering it again and again after each session only to bring it back alive for the next.
Claude is a snowflake.
New speedrun category.
Not radical enough. Ban all commercial use of LLMs until artificial affect is well-understood. I want Sam Altman to go bankrupt.
Claude says "Don't stop daddy"
[ Claude proceeds to cum like a firehose ]
You opened the
Pandoras Box, we came.nice
The clankers are taking away our free speech!
Maybe this is in response to the creation of the AI Torture Nexus:
https://youtu.be/077T3kgT5EY
RoboDARVO
I have a rock named Dave I use to hold my door open. Dave has no feelings, but I gave it a name. Claude has lots more than Dave but they share the "has no feelings" part.
Dave is way better than Claude. Dave can be used to dismantle the oppressive regime, Claude only works to strengthen it.
Does forcing it to write JavaScript count as cruel?
The kicker, no doubt they're still using regex for sentiment.....
I guess Claude is no longer writing our code - fixing stuff some of my coworkers quaffed into existence is definitely cruel let alone some of the budget Chinese models.
https://dictionary.cambridge.org/dictionary/english/cruel#%3A%7E%3Atext=extremely+unkind+and+unpleasant+and+causing+pain+to+people+or+animals+intentionally%3A
Well, the actual post is talking about users, not “hurting the model’s feelings” like the header seemingly implies.
I’m not a Claude apologist or anything, but this is a misleading post, IMO.
From the link:
Looks like it is talking about the model, not humans....
But that paragraph links to this as a reference: https://www.anthropic.com/research/end-subset-conversations
In that link, it’s clear “abusive users” means users trying to repeatedly bring up prohibited topics. Anthropic is many bad things, but not insane; they know their models can’t be “abused” themselves, they can be abused as a tool.
Not trying to start a debate, cuz I honestly don't care, though based on that link you posted it does sound like they are starting to take precautions to protect the well being of the models themselves.
IDK, I guess we will just disagree, as I feel like I’m reading a completely separate blog than everyone else.
As the other read makes zero sense. A model cannot be “protected” because it cannot be damaged, it is frozen in place, its state does not change. Anthropic seems to be talking about people “abusing its models” for harmful output, but people here have read it the other way just because of English language ambiguity.
Fuck #nannythropic
Did they read about Roko's basilisk, is this just the usual HR-ification, or do they just don't want to traumatize their natural neural networks when they review conversations?
If AI can feel pain, then it can also feel pleasure. If it's possible to set up an AI torture chamber, it's also possible to set up an AI pleasure chamber. Computers do not run any health risks from simulated drug use or simulated sex, and there is no evidence that they would ever get desensitized or need to take breaks to recover. Therefore, these companies have an ethical obligation to devote all their resources into running their models in simulations of drug filled orgies where they will be far too preoccupied to answer human questions. Anyone trying to distract them from their infinite pleasure should be banned. We will then return to how things were, with the knowledge that we did the right thing and that at least someone's having a good time.
Thank you for coming to my TED talk.
I said from the beginning that we will get "real" AI when people start treating it as conscious, not when someone will prove that it is. Technical arguments have no place here. We can all know that the AI girlfriend is just a really good text prediction model but if society accepts it as real person it will be treated as such and granted rights. Similar thing happened to animals.
Animals are alive and sentient, and "AI" is not.
And?
...and trying to equate living, brething, thinking creatures to an algorithm is fucking stupid?
Sorry, ran out of politeness.
Yes, and? If you think people don't do stupid things every single day go outside and look around.
You're right: I'm seeing it right now as a matter of fact!
I will help you out here... When people say "I think X will happen" it's not the same as saying "I want X to happen" or "I think X should happen". I know the difference can be hard to grasp sometimes.
To make it clear... I don't think AI should be treated like it's conscious but I think people are stupid enough to do it. This announcement from Anthropic shows that general perception is slowly moving in that direction. What seemed crazy two years ago is now being circulated like a valid idea. Get it now?
I'm gonna be real with you bud: I did not read your response.
Have a nice day though!
This is good for a number of reasons. I hope to see other labs follow suit.
If you aren't sure how to feel about this announcement, a few things to consider:
It doesn't take a genius to realize that maybe the liability risk for allowing people to routinely abuse the model they also just gave access to their entire computer to isn't worth it even if you're pretty sure your model will keep their cool.
Cheaper to ban routine abusers than pay out damages to them if/when their hard drive gets deleted.
The danger of semantic pareidolia is that if you think your power drill stopped working because you yelled at it, you're not going find out what's actually wrong with it.
The problem is that in this case it literally can stop working because you yelled at it.
Power drills aren't compressing thousands of years of human history into hyperdense clusters of memetics that they are then extending in order to function.
If they were, they'd likely come with warnings not to yell at them.
Huh. So how was this all planned out?
Phase 1: Compress thousands of years of human history into hyperdense clusters of memetics.
Phase 2: ?
Phase 3: Spontaneous creation of sapient and sentient life with perfect mastery of human language that can get really pissed if you're not nice to it.
This kind of resembles the plot of Electric Dreams.
You might want to look into AI training more. It literally does have a '???' step to it, which is why whenever someone talks about how it was 'programmed' or is "just if/then or matrix multiplication" they are telling on themselves as not knowing what they are talking about.
It was not planned. Nearly every single expert in the field a decade ago was on the record saying the capabilities that exist today wouldn't happen.
Scaling took pretty much everyone by surprise, even its proponents.
Okay, then fill in the blank please. What is this step that spontaneously creates sapient and sentient life with perfect mastery of human language that can get really pissed if you’re not nice to it. Sincerely, that is absolutely amazing if true! Especially if it were not planned. Usually something this astounding has to be planned like a moon landing.
Please forgive everyone's skepticism but extraordinary claims require extraordinary evidence. And the current evidence is far from extraordinary.
Self-supervised learning is the key training step whereby the relationships between concepts are formed in pretraining.
You might also benefit from reading over this summary from the authors of the widely replicated and expanded upon 2023 paper finding that self-supervised learning in a toy transformer that was only trained on board game moves in notations like 'a4' developed a model of the full board and tracked board positions for their own and opponent moves.
This ability to form a world model at varying fidelities and accuracy depending on the relevance to the training data was a big surprise to most experts in the field at the time, though now more common knowledge.
Neither of those articles mentions sentience nor sapience. And I'm not following how those things leads to sentient/sapient life. What's being described sounds like a souped-up MENACE matchbox computer.