Kinda silly title. The question whether something can be done, or is being tackled in real life, are two very different questions.

Last year when I read about AI psychosis, the state of the world was, there's been some lawsuits after people killed themselves. And we learned OpenAI's internal systems even worked(!) They flagged that account hundreds of times. It's just that they didn't do anything about it. So it took a turn for the worse.

Dunno if that changed since. I bet they're not letting anyone have a look unless there's some big lawsuit... But why argue hypotheticals from a world which doesn't exist? First thing they need to do is do something in the first place. We probably can't answer the question from the title until someone puts in some effort and tries.

No. A word predictor is not going to be able to give these people what they need. What they need is tailor made psychiatric help from a qualified physician or health practitioner.

The very reason that these language models are so "good" at enabling people to do bad things in moments of crisis is because they are designed to tell the user engaging them reaffirming statements in order to keep them on the hook spending tokens.

Edit: The absolute best we could possibly do is to flag that behavior as concerning and force a universal healthcare system to do wellness checks to get these people the help they need before the worst happens. But of course, to do that we'd actually need universal healthcare. Plus, that comes with the problem of actually tying the account to the real identity of the person which I'm pretty against.

Have you ever heard of the Radium Girls? They were factory workers -by and large women- who painted watchfaces with radioactive paint, which let the numbers glow in the dark. These workers were taught to form the bristles of these tiny paintbrushes to a fine point with their lips, like you would wet a frayed thread. Each time they did this, they ingested a little bit of radium-infused paint.

When it became public knowledge (well, more public) that radium was dangerous, the workers' bosses told them to use their fingers to make the thin points on their brushes. But this was slower, and the workers were paid for each completed watch face. So these desperate, underpaid workers kept using their lips to get through more watch faces, each time swallowing a miniscule amount of the radioactive paint that was slowly killing them.

There's a parallel here. The people who own AI companies can slap a disclaimer on their chatbots that it isn't therapist and shouldn't be used for medical advice. But when you can't afford a therapist and yet you know you need one, that free chatbot sure sounds appealing.