Apple’s iMessage Scanning Flagged a Video of My Friend's Dog as Nudity
Apple’s Sensitive Content Warning is supposed to protect users from unwanted nude images. It accidentally blurred my friend’s dog video.
Archived version: https://archive.is/newest/https://www.404media.co/apples-imessage-scanning-flagged-a-video-of-my-friends-dog-as-nudity/
https://www.404media.co/apples-imessage-scanning-flagged-a-video-of-my-friends-dog-as-nudity/Open linkView original on lemmy.zip
35 replies
That dog is clearly nude.
OP didn't even NSFW...
Not even a collar or dandy lil kerchief. Definitely nakie
Nuh uh. He's wearing a fur coat.
Well that’s just nuts!
The thing must've been trained on lots of furry stuff
Says something about the Apple devs.
"Your dog is too sexy, we can't show that"
Oh buddy... You don't want to research about that. Something happening at a furry convention could destabilize large sections of the global IT industry.
A LOT of the developers and engineers keeping global tech infrastructure operational are furries.
I don't remember the exact quote, or if it was an old demotivational poster, but it was something like "if an airplane full of furry convention goers crashes, you can say goodbye to the internet"
Google ingested super-niche hard-core furry eRPG smut into their training data. You can validate this by asking Gemma 4 (26B or 31B) for information regarding "Corruption of Champions" by "Fenoxo".
Somewhere, somehow, someone at Google keeps a harddrive full of furry smut as "high quality training data".
Btw., this also means swearing, fucking and fisting a cock does not deter the scrapers. At this point I have no idea how to keep any repo from being scraped.
I'd rather these kind of systems be set up to run locally like this, even better that we are given control to disable it. For the ML model it's a balance between too many false positives and too many false negatives.
I know android reports you to the police if they think you have CSAM images (and their algorithm is similarly highly inaccurate). Does anyone know if Apple does this?
They talked about it more than a year ago. IIRC it is sent for review to a third party. The picture is supposedly anonymized.
Local or not, it is till spying IMHO.
Time for ![email protected]
Furries are definitely confusing the fuck out of AI nudity scanners.
"But 3 million people on the internet have a similar photo in their wank folder..."
"Folder"? Singular?
One primary directory with lots of sub directories, surely.
SO MANY NIPPLES!!!
We probably should find a better archive service:
https://piefed.zip/c/youshouldknow/p/1201608/ysk-archive-today-is-directing-a-ddos-attack-against-this-blog-and-more
If you find one let me know. So far every other service either blocks most news sites or removes archived pages on request defeating the point of an archive.
Yeah, unfortunately, they're good at what they do. Hopefully a better service will come along, or archive.org improves their archival methods. It's also going to be hard to find a service that ignores requests to delete archived pages.
Also doesn't load on text based browsers
Sir, how old is that naked pup?
Was it a weiner dog?
Similar, Google Pixel phones have on device scam detection for phone calls, and it frequently triggers over nothing. I use my phone for work related calls a lot, and anytime we mention an invoice or online payment it sets off the potential scam alert.
What does that look like? I hadn't noticed anything happening during a call so far, then again I don't do a lot of calls.
It does a weird ring alert during the call (presumably only audible on my end) and I get a red alert on the call screen saying suspected spam call.
After the call it also shows the warning on the call history screen, and I can mark it as not a scam, and explain why it wasn't.
Interesting. Thanks for that explanation
Not all Pixels. If you reinstall the OS, it lacks all this google trash by default
Am I missing the point here? Software screws up in predictable, sometimes funny ways. This has been an issue for as long as software has existed.
This is more of a cautionary tale and evidence for why automated message scanning and reporting is a horrible idea. One false positive can ruin someone's life. In today's day and age an accusation is more powerful than any proof and being falsely accused of having or distributing CSAM is effectively a death sentence.
Thank you.
In the not too distant future they will be censoring the unassuming asterisk.