lemm.ee plans for mitigating image upload abuse
Hey folks!
I made a short post last night explaining why image uploads had been disabled. This was in the middle of the night for me, so I did not have time to go into a lot of detail, but I'm writing a more detailed post now to clear up where we are now and where we plan to go.
What's the problem?
As shared by the lemmy.world team, over the past few days, some people have been spamming one of their communities with CSAM images. Lemmy has been attacked in various ways before, but this is clearly on a whole new level of depravity, as it's first and foremost an attack on actual victims of child abuse, in addition to being an attack on the users and admins on Lemmy.
What's the solution?
I am putting together a plan, both for the short term and for the longer term, to combat and prevent such content from ever reaching lemm.ee servers.
For the immediate future, I am taking the following steps:
1) Image uploads are completely disabled for all users
This is a drastic measure, and I am aware that it's the opposite of what many of our users have been hoping, but at the moment, we simply don't have the necessary tools to safely handle uploaded images.
2) All images which have federated in from other instances will be deleted from our servers, without any exception
At this point, we have millions of such images, and I am planning to just indiscriminately purge all of them. Posts from other instances will not be broken after the deletion, the deleted images will simply be loaded directly from other instances.
3) I will apply a small patch to the Lemmy backend running on lemm.ee to prevent images from other instances from being downloaded to our servers
Lemmy has always loaded some images directly from other servers, while saving other images locally to serve directly. I am eliminating the second option for the time being, forcing all images uploaded on external instances to always be loaded from those servers. This will somewhat increase the amount of servers which users will fetch images from when opening lemm.ee, which certainly has downsides, but I believe this is preferable to opening up our servers to potentially illegal content.
For the longer term, I have some further ideas:
4) Invite-based registrations
I believe that one of the best ways to effectively combat spam and malicious users is to implement an invite system on Lemmy. I have wanted to work on such a system ever since I first set up this instance, but real life and other things have been getting in the way, so I haven't had a chance. However, with the current situation, I believe this feature is more important then ever, and I'm very hopeful I will be able to make time to work on it very soon.
My idea would be to grant our users a few invites, which would replenish every month if used. An invite will be required to sign up on lemm.ee after that point. The system will keep track of the invite hierarchy, and in extreme cases (such as spambot sign-ups), inviters may be held responsible for rule breaking users they have invited.
While this will certainly create a barrier of entry to signing up on lemm.ee, we are already one of the biggest instances, and I think at this point, such a barrier will do more good than harm.
5) Account requirements for specific activities
This is something that many admins and mods have been discussing for a while now, and I believe it would be an important feature for lemm.ee as well. Essentially, I would like to limit certain activities to users which meet specific requirements (maybe account age, amount of comments, etc). These activities might include things like image uploads, community creation, perhaps even private messages.
This could in theory limit creation of new accounts just to break rules (or laws).
6) Automated ML based NSFW scanning for all uploaded images
I think it makes sense to apply automatic scanning on all images before we save them on our servers, and if it's flagged as NSFW, then we don't accept the upload. While machine learning is not 100% accurate and will produce false positives, I believe this is a trade-off that we simply need to accept at this point. Not only will this help against any potential CSAM, it will also help us better enforce our "no pornography" rule.
This would potentially also allow us to resume caching images from other instances, which will improve both performance and privacy on lemm.ee.
With all of the above in place, I believe we will be able to re-enable image uploads with a much higher degree of safety. Of course, most of these ideas come with some significant downsides, but please keep in mind that users posting CSAM present an existential threat to Lemmy (in addition to just being absolutely morally disgusting and actively harmful to the victims of the abuse). If the choice is between having a Lemmy instance with some restrictions, or not having a Lemmy instance at all, then I think the restrictions are the better option.
I also would appreciate your patience in this matter, as all of the long term plans require additional development, and while this is currently a high priority issue for all Lemmy admins, we are all still volunteers and do not have the freedom to dedicate huge amounts of hours to working on new features.
As always, your feedback and thoughts are appreciated, so please feel free to leave a comment if you disagree with any of the plans or if you have any suggestions on how to improve them.
Personally I say just leave hosting of images to dedicated sites for that purpose. Your efforts are better left to dealing with how to render them. That being said, I use to be in charge of managing abuse on a site that has an average of 20 million posts a month (seriously).
The way I essentially defeated these kinds of attacks was with an image scanning service. It scans for anything NSFW and blocks it. Sometimes things would make it through but once an admin flagged it we could use that to block the users IP and account. It’s not cheap but the volume is also not huge yet for lemm.ee so it might not be too bad.
This is my opinion also. Reddit turned to shit around the time they started self-hosting. Imgur only exists because people needed a place to host reddit images.
Is there a fediverse instance of Imgur?
No, but there's nothing stopping you from using direct links from imgur, in traditional fashion.
It's a little bit convoluted, though. You have to post the image, then hover over and select "Get share links", and then pick the option for BB code (forums). This has the [img] tags at the start and finish, but importantly it has the direct link to the image file. If you use this on lemmy then it will load in the instance, rather than directing to imgur itself.
Imgur is deleting images over a certain age posted anonymously. And they might continue to decrease the number of images they keep to try to be closer to profitability. So that will be bad for longevity of content.
reddit is the new imgur. I post stuff to my reddit profile grab the image link and post here. let spezz pay the bill for hosting our images.
This is brilliant. someone should make a way for us to provide login info to reddit that will just "login" and "post" an image to some random private sub, then return the url. A browser plugin would probably do this easily.
I would love it!
I've seen people link to uploads on Pixelfed, though this is probably not the intended use case.
Not yet but I wish there was. I use imgur quite a lot and I like the idea of a fediverse version. Especially with the direction they've gone lately.
Yeah genuinely we could all be hosting images for free or cheap on several image sites. Even NSFW images and videos! And it would save our instance admins a lot of headaches and probably some cost too.
They aren't profitable, so they'll eventually go down. If no one is looking at their site, why keep it going just to serve other sights?
The same can be said about Lemmy.
But if the instance goes down, no one will care that the images in the posts are also gone.
You forgot getting the authorities involved when somebody does upload csam
It's a known tactic by trolls to upload cheese pizza and then notify the media/the authorities themselves because context doesn't matter when it comes to CSAM
The Lemmy.world team is getting some authorities involved already for this particular case. I am definitely in favor of notifying law enforcement or revelant organizations, and if anybody tries to use lemm.ee to spread such things, I will definitely be involving my local authorities as well.
How do you imagine that playing out? This isn't some paedophile ring trading openly, this is people using CSAM as an attack vector. Getting over-enthusiastic police involved is exactly their goal, and will likely do very little to help the victims in the CSAM itself.
Yes, authorities should be notified and the material provided to the relevant agencies for examination. However that isn't truly the focus of what's happening here. There is no immediate threat to children with this attack.
FBI: Whoa that illegal
Admin: Ya
FBI: We're going to look for this guy
Admin: alright
END ACT 1
This isn't something the FBI have much involvement with. The FBI deal with matters across states.
This isn't America, where you have a bunch of separate states unified under one American government. People haven't been posting porn to lemm.ee. People have been posting porn to other instances, which has seeped through to lemm.ee.
Getting the Estonian law enforcement involved is like trying to get the Californian government involved in dealing with a problem from Texas. Estonian law enforcement have no jurisdiction over lemmy.world or any other instance, and giving them an opportunity is only going to lead to locking down lawful association and communication in favour of some vague "think of the children" rhetoric. And, like I say, it won't do anything to curtail the production of CSAM as the purpose of this attack has little to do with the promotion of CSAM.
Frankly, it could easily be more like:
Meanwhile, if lemm.ee continues its current course of action, yet someone notifies law enforcement:
It really matters how and when the problem is presented to law enforcement. If you report yourself, they're much more likely to take action against yourself than if someone else reports you. It doesn't do yourself any favours to present your transgressions to them, not unless you're absolutely certain you're squeeky clean.
At this stage and in these circumstances, corrective action is more important than reporting.
You're assuming that no American user saw any of the content. I think the FBI could absolutely get involved if the content was seen by anyone in the US, let alone by people in more than 1 state. I'm not going to pretend to be an expert on child abuse or cyber crimes but the FBI devotes massive resources to investigation of crimes against children and could potentially at least help other agencies investigate where this attack originated from. And if the FBI were able to determine that the attack originated from the US, I assure you the DOJ is far less kind to people who possess, commit or distribute that type of horrible child abuse than they are to rich old white men who commit a coup. You're kind of acting like this is just another DDOS attack rather than the deliberate distribution of horrific images of child abuse to a platform that in no way encourages distribution of child abuse material.
Anywhooooo the problem was much worse on lemmy.world since they were the main target of the attack. Does anyone know if they reported it?
Local authorities will be the contact point of the admins (or authorities of where the servers are hosted). They'll investigate what they can and then ring up euro/inter/whatever pol as necessary to have other forces handle stuff in their respective jurisdictions. Cross-border law enforcement isn't exactly unchartered waters, they've been doing it for quite a while.
As to the current case the ball is clearly in the field of lemmy.world admins and their local authorities (Germany? Hetzner, I think, as so many) as they're the ones with the IP logs. Even if the FBI gets a tip-off because an American saw anything they're not exactly in a position to do anything but go via Interpol and ask the BKA if they'd like to share those IP logs.
For step 6 - are you aware of the tooling the admin at dbzero has built to automate the scanning of images in Lemmy instances? It looks pretty promising.
Yep, I've already tested it and it's one of the options I am considering implementing for lemm.ee as well.
It's worth considering some commercially developed options as well: https://prostasia.org/blog/csam-filtering-options-compared/
The Cloudflare tool in particular is freely and widely available: https://blog.cloudflare.com/the-csam-scanning-tool/
I am no expert, but I'm quite skeptical of db0's tool:
I'm no expert, but my belief is that open tools are likely to be hamstrung permanently compared to the tools developed by big companies and the most effective solutions for Lemmy must integrate big company tools (or gov/nonprofit tools if they exist).
PS: Really impressed by your response plan. I hope the Lemmy world admins are watching this post, I know you all communicate and collaborate. Disabling image uploads is I think I very effective temporary response until detection and response tooling can be improved.
you make some good points. this gave rise to a thought: seems like law enforcement would have such a data set and seems they should of course allow tools to be trained on it. seems but who knows? might be worth finding out.)
Tbh I'm kind of surprised no government has set up a service themselves to deal with situations like this since law enforcement is always dealing with CSAM, and it seems like it'd make their job easier.
Plus with the flurry of hugely privacy-invading or anti-encryption legislation that shows up every few months under the guise of "protecting the children online", it seems like that should be a top priority for them, right?! Right...?
I replied to the parent comment here to say that governments HAVE set up CSAM detection services. I linked a review of them in my original comment.
This seems like inflammatory bait but I'll bite once.
Good info! Fwiw, I wasn't intending for it to be "inflammatory bait", but a jab at the congresspeople who use "for the children" as a way to sneak in bad legislation instead of actually doing things that could protect children
I'm not sure I follow the suggestion.
All of which is to say...
Law enforcement DOES have datasets, and DO allow tools to be trained on them... I've linked the resulting tools. They do NOT allow randos direct access to the data or tools, which is a necessary precaution to prevent attackers from winning the circumvention race. A Red Hat or Mozilla scale organization might be able to partner with NCMEC or another organization to become a detection tooling partner, but db0, sunaurus, or the Lemmy devs likely cannot without the support of a large technology org with a proven track record or delivering and maintaining successful/impactful technology products. This has the big downside of making a true open-source detection tool more or less impossible... but that's a well-understood tradeoff that CSAM-fighting orgs are not likely to change as the same access that would empower OSS devs would empower CSAM producers. I'm not sure there's anything more to find out in this regard.
gtk. thanks.
If you have publicly available detection tools you can train models based on how well stuff they generate triggers those models, i.e. train an AI to generate CSAM (distillation in AI lingo). It also allows training of adversarial models which can imperceptibly change images to foil the detection tools. There's no way to isolate knowledge and understanding so none of it is public and if you see public APIs they're behind appropriate rate-limiting etc. so that you can't use them for that purpose.
The neat thing is that it's all much easier as lemm.ee doesn't allow porn: The filter can just nuke nudity with extreme prejudice, adult or not.
It seems promising but also incomplete for US hosts, as our laws do not allow deletion of CSAM rather it must be saved and preserved and sent to a central authority and not deleted until they give the okay. Rofl.
I also wonder if this solution will use PHash or other hashing to filter out known and unaltered CSAM images (without actually comparing the images, rather their metadata).
i didn’t quite know that and yet, it doesn’t surprise either.
The flip side of the argument is that if you also host the media you are not at risk of having broken links. I’ve seen a number of long running forums that had post bodies that contained external images that are now broken.
Of course an argument can be made that the only reason that those forums have lived for so long was due to not having costs associated with hosting media.
That's no worse than a reddit link getting borked because it's been cross-posted and someone managed to kill the original link with a DMCA notice.
I would say that is a different issue. DMCA could go to whatever external host as well so that doesn’t change.
My argument was about putting faith in external providers to stay alive to continue hosting media. You can also get in a situation where an external provider decides to do a mass delete like what Imgur did this past summer.
A post getting removed because someone threatened legal action is not the same as using an image host that goes under because no one visits their site to see their ads to pay for hosting it or because they arbitrarily purged their content or changed their link format like imgur has. Unless Lemmy hosts it's own images it will be at risk of being purged like has happened many times over.
I get we don't trust these third party image hosting sites, but if it's that or having local images that can potentially bring down instances, I'd say that's a no brainier of a compromise.
These upload sites like imgur automatically handle image detection and take the load off smaller servers. It seems like a perfect solution got now
There is a privacy and tracking concern with loading images from 3rd-party hosts vs lemm.ee hosting or re-hosting them.
Please please do not implement an invite system.
The success of a forum like this depends on people being able to join and express their thoughts freely. Reddit and digg would never have gotten where they are if they had a closed system.
I almost didn't join lemmy because the first two instances I heard about (lemmy.ml and beehaw) had closed registration. I think I applied and then forgot about it for 2 weeks. Thankfully I saw a post about lemmy on reddit yet again and finally found an open instance.
Don't let the actions of a few scumbags ruin a good thing for everyone. You'll be giving them exactly what they want.
I agree that users should be able to join Lemmy freely, but I think it makes a lot of sense to try and spread users out more between instances - this spreads out the responsibilities between more admins, spreads out the load between more servers and also reduces the chance of a single point of failure for the whole system.
It's clear that there are seriously vile people out there who want to cause huge amounts of damage to Lemmy, and if we have unlimited growth in a few selected instances, then these people only have to target those specific instances for maximum damage.
In a perfect world, none of this would be necessary, but then again, in a perfect world, we wouldn't need a decentralized platform in the first place.
Thanks for responding!
I agree that it's best for the lemmyverse.net if there are many big instances too.
Unfortunately, the concept of the fediverse isn't as easy to understand. The average newcomer (who mostly just wants to consume content and occasionally ask a question or two) starts off by interacting within their instance, and it takes some time to figure out cross-instance communication (there are still posts about this on the nostupidquestions-type communities). For such users, landing on a small instance means they'll poke around the Local active posts, think that "this forum is dead", and never return.
Like reddit, having a large userbase on lemmyverse is important to keep the conversation interesting (see https://i.imgur.com/4tXHAO0.png). Reddit has provided lemmy with a huge shot at success by injecting a large number of users. But if I'm being honest, the conversation on the lemmyverse isn't as diverse and engaging as it is on reddit yet. This isn't self-sustaining yet. I can point to 2 pieces of evidence to support this:
Using Voat as a (imperfect) proxy - I don't know if there are official stats of Voat, but the best dataset I've seen for Voat (https://ojs.aaai.org/index.php/ICWSM/article/download/19382/19154/23395) has 16.2M comments in 2.3M submissions from 113k users. Voat was shut down for lack of funding, but even in its heyday it wasn't exactly thriving - many people on Voat were united in their toxicity and it never really got going. Compare these numbers to the lemmyverse which has about 100k active users over the last 6 months. If the fediverse is to grow beyond "that niche forum for nerds", this userbase isn't enough.
It's already clear that the number of active users is decreasing - since mid-July, the number of monthly active users has dropped from 70k to 50k. This is expected (bunch of redditors who joined in June, poked around and said hi and left), but it means if the lemmyverse wants to have any chance of succeeding long term, you can't alienate new users now.
The approach I've been advocating since the beginning of lemmy is:
The way federation works now, it's still disadvantageous to be on a smaller instance (discoverability of new communities is harder, syncing posts/comments isn't always fast, it's hard to know which community is more active. Many of these can be fixed with changes to activitypub and lemmy protocol, but in the meantime, sending casual users to small instances means they'll likely never return.
So to sum up, I think there should be an avenue for casual users to join the biggest instances, even as we encourage people to move to smaller ones (either targeting those who are more tech savvy, or those who have already been on Lemmy long enough to know how it works - I myself was on Lemmy.world and switched to this "smaller" instance).
Anyway, you're the admins here and I have no say over what you eventually do. I'm just hoping you'll consider the practical realities of user behavior - everyone wants what's best for the fediverse in the long term.
https://github.com/Fmstrat/lcs
My experience is the opposite, but that may be instance dependant
Active users stats are the same on every instance for communities
If I may, lemm.ee is now the second biggest instance. Redirecting people to register on local instances (feddit.country) or generalist ones (reddthat.com, Lemmy.today, discuss.online etc.) couldebe reasonable to make those ones grow as well.
I agree that there should be a clear lists of instances open for registrations, but that probably needs to wait for the dust to settle a bit beforehand
I posted a long reply above (direct link https://lemm.ee/comment/2929349).
While I understand your concerns, this instance has gotten a fair bit larger and will start to suffer the same issues that lemmy.world does if registrations aren't curbed. It can't grow infinitely. That just isn't feasible for one server. Having closed registrations on lemm.ee doesn't stop anyone from signing up on different instances. A solution might be to temporarily limit registration here in some way, and for the devs and instance admins to find a better way of helping new users choose an instance. The initial sign up process was confusing, and could be streamlined to make it easier for people to choose an instance. In the long term, enhancing the way federation works so users who do sign up on smaller/newer instances don't need to be lemmy savvy to find content would also help alleviate that type of issue.
I posted a long reply above (direct link https://lemm.ee/comment/2929349).
i’m on the side of “no, rampant froth without proper tools and whatnot is a recipe for something we don’t want to eat.
i get your point but some folks aren’t that put off by it, assuming they can ask for an invite and it does t take ten years. i had to work at it a bit over on reddit but i took my time and just wrote about the difficulties and in a couple weeks hey, i got an invite. i’d prefer a nicer community once i’m in to a quick and easy entry but it sucks thereafter (or is just chaotic and unhappy periodically). it’s like your house. do you just let everyone in from fear of being lonely? probably not. probably, if you’re not a outlier, you’ve taken steps to make it a bit hard for anyone not invited to enter. and it makes your home a better place to be.
All of this seems good to me except 4 - I hate the thought of any instances being invited only. I'd much prefer it was just a verified user approach (even just an email) with a waiting period for doing things like posting images. Maybe even limit newish users after that period to a small number of image posts a day.
Making an instance feel like a club is going to turn off a lot of people. For sure do what you need to do, but I hope you can avoid that one.
Strongly agreed. Lemmy needs to grow. I badly miss many smaller communities that are only viable with Reddit's size. Making prominent instances invite only (or requiring approvals or closing sign ups entirely -- as some other instances have done) is just going to hurt Lemmy as a whole.
Treating new accounts with a lot more scrutiny makes sense to me. We could require the first few comments to have mod approval to even show them (probably more of a per community setting since it would likely have to depend on community mods), restrict images for some period, have more aggressive content filters on young accounts, etc.
Yep, that's the right idea. Can't be anything that's too labor intensive for the mods, that that's the right thinking.
I agree. The lemmyverse still need to grow to have any chance of lasting beyond the next year or so
If volunteer admins are at their limits, tools to enable admins to manage larger communities needs to come before further growth. Yes, lemmy needs an order of magnitude growth to be able to seriously compete on content, but outgrowing admin capacity is not a sustainable path.
yes. very sound thinking there. many problems in life can be skipped by some forethought and timing. don’t invite folks for dinner before your kitchen is fully built.
thank you for your work sunaurus, and i'm sorry you had to sort through this
(particularly annoying though, as i never got around to adding a user banner; and i had one in mind as well. i wish there was some way to externally host avatars and banners)
I'm going to be a part of an invite only community?! Of course, given the circumstances, this is pretty fucked. But I feel kinda fancy right now.
Thanks for all you do on lemm.ee
I left Twitter before musk, when the security chief said that they know they have CP but they were doing nothing.
I can forgive a measure that doesn't work as expected or at 100% but not the inactivity.
Therefore I'm agree with any measure you think it can work despite any inconveniences for me.
Sorry for any misspelled or wrong word, English isn't my main language
Regards and thanks for all your efforts.
Your English is flawless and your sentiment is echoed. The last thing we should do is to ignore the problem.
This has been a great instance since day one, and it's good to see you once again being so proactive. Thank you for the update!
There are downsides with all kinds of moderation, but ultimately most of us accept that the internet can't function as a true free-for-all. Absolutely in support of whatever you feel is necessary to keep the server safe, but please watch out for yourself too and make sure you're asking for help where needed.
p.s. anyone reading this who doesn't donate to the server yet, here's a reminder that that's a thing you can do.
Could you post a guide on disabling the local image cache? I compile from scratch so I’m not afraid of making changes in the code, I just don’t really know rust. I shut down my personal instance and this would allow me to turn it back on.
This should do the trick: https://github.com/LemmyNet/lemmy/commit/c74b92e5fd3b6e4b815cb76768bb8b042784ef6c
(I am fairly certain that this is the only way external post images get saved locally, but if I find additional ways later, I will update this comment)
Sounds like the old karma requirements some reddit subs had. While I'm not against that, it would restrict locally registered users more so than others who are posting on lemm.ee communities when their host instance has no such system in place. I'm aware that if they post images those would be uploaded to their home instance and linked here with the patch you mentioned above, but the downside is that local users might feel inconvenienced more so than others. Not saying it's a bad idea though, if we are thinking from a "protect lemm.ee" angle first and foremost.
You might want to reach out to the dev of Sync for Lemmy, ljdawson on [email protected], he just implemented an anti-NSFW upload feature in the app to do his part. Essentially, Sync users currently can't post any kind of porn. While I don't think that the CP spammers were using his particular app, or any app to begin with, I do think it's a neat feature to have, but would make much more sense to run server-side.
but what about normal, legal, NSFW material?
For now it's all or nothing. Better safe than sorry.
Not allowed on lemm.ee in the first place. Well, you can see NSFW posts and subscribe to everything on lemmynsfw.com but you're not supposed to post any porn from a lemm.ee account.
Policing NSFW is a whole can of worms, it makes sense to leave it to specialised instances. They can nuke political drama from orbit, we can nuke nudity from orbit, both saving mod bandwidth to do the other thing right.
Got to be honest, having an invite based system and locking certian features behind age of accounts, karma, etc seems like the opposite of the freedom everyone promised me the Fediverse represented when we moved over.
I personally don't really care about images and would prefer image uploads just stay deactivated and we operate as a text only forum but with open membership.
Leaving image uploads completely disabled would also be an option to fight this particular type of attack, but there are also other issues with open registrations. For example, while our sign-up captcha seems to be preventing automated registrations, we are still having to ban advertiser accounts almost daily. I think an invite system would really help to reduce sign-ups by any kind of users intending to abuse the system.
I'm all for an invite-based system, although we will need some way of combating 'invite trees', where one bad actor invites several others, who subsequently invite an exponentially increasing number. A reasonable delay on the invite allowance would go a long way, I think.
I have to say that an invite based signup system makes my toenails curl backwards. IMO this will let instances die out slowly. I didn't know anyone using lemmy and just stumbled upon it. ppl like me wont ever be able to join an instance if it is invite only.
Don't misunderstand me: I do understand how critical it is for the operators of instances to protect themselves. Lemmy is a rather young project and still needs better admin tools. However, there are some good discussions happening on GitHub. Untill the operators and admins have the tooling to protect themselves, I see disabling img upload as preferable. It also took reddit some time to allow uploading images, instead of linking them.
I 100% agree! An invite-based system means that a new user has to find some way of contacting someone in order to request an invite. I think that only allowing X posts per day for e.g. the first week or 2 for new accounts would be a way to combat companies and spammers. Not allowing images or limiting image posts for new accounts, and using automated CSAM detection methods, which I understand are in the works, seems to be a good way to combat that problem.
That's a fair point. It's not a situation with an easy answer.
I very much agree, invite-only systems are a bad idea for this reason.
I like almost everything on this plan, except for the last 2 items. The account requirements for "extra activities" best be chosen carefully as to not encourage the good old "karma farming" that we got away from in leaving Reddit.
And the ML thing for recognizing NSFW is also something to be carefully considered. Too strict and it gets annoying with false positives, it can restrict posting actual content, and too lax won't make a difference for the people actually looking to circumvent it. I think a "vetting" system like the previous item could be better in the long run, in only letting "trusted" people upload content.
I hope there is another option besides just deleting images indiscriminately. I run several comic strip communities and it would be a shame to lose all the posts and work I've put in.
What about implementing Imgur or something similar, assuming they scan for CSAM on their end. For example I often use the Lemmy iOS app and I noticed that all my image uploads using the app are through Imgur.
@TWeaK is correct, I am only deleting our copies of images which are already hosted on other instances.
As for imgur (or any other external image host), such images have always worked on lemm.ee. For example, this is hosted on imgur:
In addition to using external images in comments, you are also able to submit posts with imgur images, and they will get embedded directly into the Lemmy UI.
Thank you.
You wouldn't lose the posts you've made, rather the posts you've made will be hosted from one instance, rather than all of them.
You're a lemm.ee user, if you upload to a lemm.ee community nothing will change.
If you upload to another community, then normally you're post would be uploaded to lemm.ee. This would then be federated, and users from other instances would load the same content, but it would be delivered by their own instance.
The change refers to things beings hosted only in your host instance. Thus, a lemm.ee user may load content from a lemmy.world server more often. Normally, lemm.ee would copy the content to its own servers and direct its users to that, but now everything will go to the host instance.
The only thing I'm not sure about is who is the host instance? My understanding is that the host instance is that which the user belongs to. Thus, if a lemm.ee user posts to a community in lemmy.world, techincally the federated host instance is still lemm.ee - it's about the user, not the community. But with all this I'm not sure.
That's my understanding too - if you're a lemm.ee user, and you want to upload to [email protected], then when you post, you're actually posting to lemm.ee/c/[email protected]. This rule change means that - for the moment - lemm.ee users can't upload any images directly when making a post, irrespective of where the community is. They'd have to use an external host like imgur.
Side note, but thanks for posting all the comics. I see them all the time.
I prefer a more text based main post experience so this is gonna be good for me. Reddit used to be a fantastic discussion forum until every single post on /all was either an image post or video post. I wish there was a way to completely disable media posts so I could just view discussion posts.
You could probably find an instance that's dedicated to this
Lemmy admins need to do whatever it is they can to handle CSAM if and when it arises. Users need to be understanding in this because as I’ve argued in other threads, CSAM itself poses a threat to the instance itself, as it poses a threat to the admins if they cannot clean up the material in a timely manner.
This is going to likely get weird for a bit, including but not limited to:
I just want folks to know that major sites like reddit and facebook usually have (not very well) paid teams of people who’s sole job is to remove this material. Lemmy has overworked volunteers. Please have patience, and if you feel like arguing about why any of the methods I mentioned above are BS or have any questions reply to this message.
I’m not an admin, but I’m planning on being one and I’m sort of getting a feel for how the community responds to this sort of action. We don’t get to see it a lot in major social media sites because they aren’t as transparent (or as understaffed) as lemmy instances are.
well said and absolutely correct. thank you.
Here is an alternative Piped link(s): https://piped.video/KuY2gKUxFbg?si=XIUIwEBj2p_M7uHA
Piped is a privacy-respecting open-source alternative frontend to YouTube.
I'm open-source, check me out at GitHub.
Thanks for keeping the community updated and for all the work you put into maintaining it!
Great plan, love seeing someone try to combat this!
Thanks for this.
As others have pointed out, perhaps just sticking to external hosting for images would make the most sense as long as the costs are manageable.
If possible, could you tell others how to apply this patch to their own server?
See this comment: https://lemm.ee/comment/2901943
How would one go about implementing this change on their server?
Seems like a good plan. I have been very impressed with your approach to administer ing lemm.ee.
Regarding the planned invite system, what would be the consequences of inviting a malicious user? I would think it would be hard to enforce any consequences simply because of the open nature of lemmy as an ecosystem.
That sucks, but hopefully something good can come out of it eventually. Like better mod tools...
I've searched what CSAM means, and I'm flabbergasted! Doesn't the authorities catch these criminals?!
They sure try, which is why it is such a difficult issue. If an instance were to hold onto it for too long, it would likely get caught in their cross hairs and taken down.
🎯
The bans seem to only be targeting those posting csam no? I could personally live in an anti-cp echo chamber
I think you make a fair point, but Lemmy has one major strength that lobste.rs does not have here: Lemmy is federated, so you can split up your social graph between multiple instances, or as many people do, you can even host your own instance just for your own inner circle.
If I may, lemm.ee is now the second biggest instance. Redirecting people to register on local instances (feddit.country) or generalist ones (reddthat.com, Lemmy.today, discuss.online etc.) couldebe reasonable to make those ones grow as well.
I agree that there should be a clear lists of instances open for registrations, but that probably needs to wait for the dust to settle a bit beforehand
lobste.rs is an interesting case study. On the one hand, it sucked to want to join and be unable to! I was in that boat for a while. And it is also disappointingly low-volume; it can be hard to get much of a discussion going just because the user base is so small.
On the other hand, when a discussion does get going, it has easily the highest signal:noise ratio of any technology message board I've ever participated in. Very few low-effort posts, and a high percentage of well-thought-out, respectful conversations.
I'm not saying I think lemm.ee should follow this model, but it's not without its merits.
These are great ideas especially the ability for users to invite others. I think it’s also a good way to get new people into the fediverse since inviting someone will have them easily know what instance to go to.
Will you submit all these features to the official lemmy backend too?
Yes, my goal is to submit PRs to the main Lemmy repo with all of these changes
It is now impossible to add an avatar or banner to profiles because the only way to do so through the UI is uploading to the instance. There’s no way to add an external URL. Just wanted to point that out in case it wasn’t intentional. Very understandable if that’s something we have to sacrifice for the time being.
Edit: I noticed that images will upload to the account's home instance instead of the community's home instance. This means that one workaround for the time being to change your lemm.ee community's icon and banner is to create an account on another instance and then add that account as a moderator to your lemm.ee community. You can then use that external account to change the icon and banner of your lemm.ee community because images will be uploaded to whatever instance your account is on instead of lemm.ee.
Nor sure about links but avatars and banners were intentional because those would be images, and all image uploads are suspended for now.
Might be useful to adjust the field themselves so instead of looking for a local image, they can pull in something from an external URL (hosted on imgur). That way at least avatars / banners will still work without the onus being on the server.
Hopefully they sort out all this stuff eventually
Would be easily doable, I was able to do it on my account here since the user settings config just contains a link to an image which can point at pictrs or at an external image hosted elsewhere so using the API (in my case using a third party app being built by my friend) you can replace avatars with externally hosted images, I made an issue on the lemmy-ui GitHub to hopefully get that feature integrated officially.
Unfortunately it's a side effect of disabling uploads, it is possible to have externally hosted avatars (field is just a URL linking to the image) but isn't officially supported by lemmy-ui yet, hopefully it can be in the future though, I made an issue for it on the GitHub to hopefully get it supported (linked in my other comment).
I understand that admins need to take whatever measures needed to protect themselves from legal pursuits
At the same time I hate to see the promised federated network revert to what commercial platforms have become, karma and account age requirement, phone and identity verification , forced 2fa and what not.
Maybe lemmy should implement a shared database whereas if an admin of an instance marks a post as potentially illegal, it gets replicated to other instances automatically and gets in queu for deletion.
The promise of the federated network is only really that any one single instance operator can not force changes upon the whole network. If users are unhappy with any instance, they always have the chance to move to another one or host their own, while still retaining access to the same federated network - this is not possible with existing centralized platforms.
🎯
Couldn't agree more - that's why we came over here, surely? Hope you are able to find a workable solution to the problem.
I think desperation of devs, admins and users is exactly the sentiment the trolls were trying to elicit. Lemmy is a young project, and this is one of many hurdles it'll need to overcome on its path.
I like the idea of removal flags propagating through the network, at least as an additional signal. Forcing removal everywhere on a single removal signal on a single instance would probably be too jumpy (e.g. a sfw instance might prevent any instance from hosting nsfw content), but some configurable rules and thresholds paired with removal reason context might significantly automate the process.
The reason I especially like this suggestion is because smaller instances can benefit from any automation that is affordable by larger ones.
This opens at some terrible abuse, just open a malevolent instant and start flagging all the content you don't like as illegal
While I share this very same feeling, I also recognize there are reasons why commercial platforms have done what they've done, I don't think they're inherently evil, they just had to face the very same problems we have
Thank you for the update. I appreciate your efforts in moderating and keeping us informed.
I think the only sustainable option here is to keep media on the instance it was first posted to and every instance managing their own stuff.
If it gets too crowded close registrations and another instance grows.
Hey there! Why not talk with the main lemmy developer to try and integrate such a content blocker directly into the lemmy stack so that it’s easier to implement for smaller instances? Thanks for keeping this instance up and runnin’! Cheers!
https://github.com/LemmyNet/lemmy/issues/3920
Great plan!
A karma system is sounding pretty good right now... /me lifts shield and ducks
Even if it's just a a limited tiered system with numbers to obsess about. Level - 1 browsing rights. Graduate to level 2 after 5 days and total of greater than 30minutes of logged in activity
Level - 2 commenting rights. Limited to 10 comments daily for 5 days.
Graduate after at least 3 comments, total upvote count >+3, and 5days.
Level 3 - posting rights. Limited to 3 posts daily for 5 days. Unlimited commenting.
Graduate after 5d and total upvote count >50
Level 4 — image posting rights. 10 images per day max
Graduate after 2 weeks and total upvote count >100
Level 5 - you've made it, everyone is equal here. Entry level users are still enjoying and growing into the community. No need to be a tool about trying to get more karma / points and number of bots / temp accounts / total losers should be minimal by this screening level.
Bots would just abuse such a karma system, wouldn't they?
It would attract the karma farming bots that reddit has. Any website that has a privilege system causes accounts with more privilidges to be worth more to buyers.
Yes for a karma point system. People will buy karma's in the thousands, but how much are people going to pay for a max score of 5 that is just there as entry level screening buffer. I don't imagine there would be sufficient value to go through the effort of farming these kinds of accounts.
Not many systems they can't, but my outline is a lot of hoops to jump through and also has a significant time gate which limits rapid attacks. Also, there's enough steps for opportunities for pattern detection to sniff out bots here as well
lemmy's code already does it. person_aggregates keeps track of post_score and comment_score. It just isn't displayed on lemmy-ui. A bot or new code can look at these values.
Welllll, my favorite discord does have a Recently Joined / New Member role. You need to post 100 text messages and have been there for 3 days before you have the ability to post images and access to the more spicy and sarcastic chats.
I think it would fit Lemmy well. Seems reasonable to lurk around a bit a first before dumping a bunch of pics onto servers right when you sign up.
I didn’t even know there was an option to load images directly from the source instance instead of caching the content locally. I know it’s a resource issue and it can slow things down a bit for users, but I think ultimately it should be done that way by default, to mitigate exponential propagation of illegal content. Wasn’t caching the main reason why lemmy.world preemptively blocked piracy communities?
That, or admins should be able to selectively choose what communities to cache content from, like maybe the ones where they can confirm there is active moderation.
Privacy-minded users want caching because otherwise it means they're connecting to multiple (possibly malicious) websites instead of just lemm.ee (someone made a post that would grab your IP and show it to you, for example). It's difficult.
Good point. I was imagining users grabbing content from the source instance via their local instance as a proxy, which would hide their info. Obviously I don't know how the backend works, so if the alternative is direct connectivity exposing your info, then yeah that's definitely something to think about.
Can we get image upload back?
I would appreciate having it for avatars and banners. I was going to change mine and couldn’t figure out why even the smallest time size was “too large. Would it be possible to have an avatar be a link for a hosted picture somewhere else or is that too much of a security concern?
Yeah it sucks that we can't upload images. Tons of people from other instances upload them to the communities, only we can't.
@[email protected]
I wonder how hard it would be to fund a full time staff to review content. That's how other platforms do it.
Other platforms also use armies of unpaid volunteers to do it. There are various methods, and with this being an entirely volunteer run an financed platform I really doubt if they is feasible. In the long term I like the idea of using technology to improve detection and moderation even if that requires some development commitment.
Overall it's a tough situation to be in. I feel a combination of account restrictions would be a way to mitigate the majority of these low quality troll accounts who get verified and then immediately start spamming.
Having images uploads tied behind user metrics such as interactions, time since creation, upvote / downvote count etc I feel would be a good indicator of a "real" user. You'll always have bad actors coming in causing issues, but at least making new users jump through hoops will make this process slower.
Closing registrations temporarily to add in extra mod features is fine, but leaving it closed and switching to an invite only system feels like it's going to slow adoption (unless in the request an invite form it's explicit that the request will be processed quickly, people will just move on otherwise)
make numbers part of it. these folks tend to come in waves. when there’s a sudden influx for no reason, it should be a flag to keep a close eye on all of them for awhile.
A way to deal with false positives of an ML NSFW scanner would be: Once per day, each user can "overwrite" the scanner. If a user is caught abusing this, they get banned.
This is an interesting idea. So if I’m understanding you correctly the workflow would be like this:
user uploads 4 images.. 2 are flagged as CSAM.
user overwrites the flag on one image, asserting that “no, this isn’t CSAM”
in other sites, I’ve seen this work by the content remaining hidden except for the user until a team reviews it. If the team agrees, it’s allowed on the site. I think this is different from what you are describing though. I think you’re suggesting that the content stay online after the user overwrites the flagging, but then a mod will later double-check to see if the user was indeed trustworthy.
I only worry that an untrustworthy user will keep the content online until a mod reviews it, increasing the time the material is online and increasing the risk. It would be difficult to argue that “this was done in the interest of user satisfaction, even though it means that more CSAM got out”. Ultimately I don’t know how many people want to argue that to a judge.
From the OP, it seems the filters don't flag CSAM. They flag any NSFW. That said, keep in mind that the filter would also have false negatives, so if people want to slip NSFW though, they might be able to do it through the filter even without such option.
But I don't mind the content staying hidden until a mod reviewed is in such cases. The false positive rate of the filter would likely be small, so there wouldn't be too many things that need review.
I think the images should never be cached from other instances in the first place, that is a huge oversight in pictrs since not only does it have the potential to cache unwanted content but also causes the images hosted to rapidly accumulate which isn't ideal as it increases storage requirements which is unfair to people who want to self-host a personal instance. Hosting a personal instance should not have monstrous storage requirements or serious liability risk due to caching all images automatically, it should only cache what is uploaded to the Instance like profiles and banners, and posts that include images from the Instance.
I have reservations about allowing fully-invite based registrations on lemmy instances. While I do think it might be good to have invites as a way for users to skip filling out an application I don't really like the idea of requiring them like Tildes does, makes it feel like an elitist exclusive club of sorts having to beg for an invite from users. I don't think it should be an alternative to application-based registration, but rather a supplement to it, if someone can get an invite from users that's great but if not they should still be able to write an application to join, this could be extensive and also lower priority since you could get invites but should still be an option available.
Account requirements really depends on what they are and what they restrict (also who on the instance is allowed to impose restrictions). For example on instances with downvotes enabled I think score/upvote requirements are a bad idea since it essentially means that people who disagree are locked out, like on Reddit with karma restrictions, I do not support this, it creates an echo-chamber where unpopular opinions. It'll also lead to upvote farming if there are negatives due to having a lower score.
Comment or post requirements would just lead to post or comment farming similar to vote farming, though it's not as bad as score-requirements since people making posts and comments naturally (whether they are liked or not) can't be taken away by other people based on opinions (only if they break the rules and get posts removed, which isn't even remotely similar since they broke the rules).
Limiting image uploading is a fair requirement in my opinion since uploads can be particularly harmful if the uploads are malicious, and also uploads aren't really needed since people can externally host almost all their images without the need for uploads.
When it comes to DMs and restrictions around them I feel like that should be up to individual users to decide to allow private communication from certain users or not, or even to allow DMs at all, this shouldn't be something globally applied to people, maybe it could be a default in User settings and have a requirement set by the Admins but people should be able to turn it off if they don't care or want to accept messages from new users, I know I certainly will, I hate being nannied when it comes to who's allowed to send me messages, IMO Annoying or uncomfortable DMs are a fact of life and I prefer to deal with issues when they happen rather than block anyone who's a new user that might want to talk to me, it's one of the things I hated that Reddit does without giving me the option to opt out and receive messages from everyone.
I think having a Machine-Learning based system to identify Malicious images is actually a pretty good idea going forward, I know how some people feel about AI and Machine-Learning but I think it's probably our best defense considering that none of us want to see it, it might have False positives but I'd rather than than to allow CSAM to live here. Ultimately the choice is have ML scanning or Disable pictrs here, I think ML is the better option because people are going to want to have Avatars and without pictrs that isn't possible (unless Lemmy adds support to the UI for externally hosted Avatars and Banners).
Once again, thank you for this wonderful instance and I'm glad this is my home.
My impression was that this was how this worked from the beginning, but apparently that's wrong. I thought the host instance (that is, the instance of the user making the post, not necessarily the instance of the community) would be the host of the image. Instead, it seems like instances share images and whatnot between themselves, to distribute the load to their own users.
Maybe this core principle is flawed. It should definitely be reviewed, anyway.
Has there been any developments on the Github in regards to all this? Really, the only things that will solve this long term are proper and granular moderation tools.
Might I suggest banning reported users? I think with a combination of users reporting posts for rule violations and mods and admins confirming and keeping them banned, it could be a better alternative for the time being.
We already ban users, but this can only be done reactively, and for something as severe as CSAM, I think it's really important to have preventative measures in addition to reactive measures.
it's been 4 months-- any update on this?
Out of curiosity, how were the CSAM images discovered? Whats stopping anyone from creating their own xyz instance and post where nobody can see?
By anyone browsing new that would see lemmy.world's shitpostint community, but anyone who made their own instance (assuming not federated yet) would essentially be in their own dark web, so there's no way for anyone else to see/cache such images
There probably is at least one instance that isn't federated that's dedicated to csam. These were uploaded to lemmy.world as an attack, and because of the way federation works some of the content is then downloaded onto the servers of other instances when it's viewed or interacted with from a federated instance.
You need to update the site to reflect these changes - I was trying to upload a banner and got the incorrect error "image too large"
Going from completely open to invite-based is what I'd call overcompensation (you're overestimating how willing people will be to invite)
So, Lemmy.World images seem to be 're-federating' here. I couldn't find any news items over there, but... did the CSAM issue finally get patched at the Lemmy software level?
Can you give an example of a federated image?
Sure. Here's a couple posted in the community I run here-- [1], [2].
They went offline for about a week I think, sometime after both LW and you shut down image uploads. The images above began re-federating just today.
Ah, those images have not actually federated to us, they are hosted by lemmy.world.
To be clear: image posts by users of other instances will work fine, but I have modified Lemmy code to no longer make local copies of the images in such posts. Your browser is fetching the images directly from lemmy.world servers when you open those posts.
Okay, thanks for the correction! I'll make sure to heed that meaning of "federated" in future.
Still, something obviously happened on the LW-side today with regard to the pics, hence my curiosity. Maybe the news will show up later, over there.
Hi there! Looks like you linked to a Lemmy community using a URL instead of its name, which doesn't work well for people on different instances. Try fixing it like this: ![email protected]
so how do I upload an avatar?
Well as always users that did nothing wrong are the ones that suffer. I think banning images is overkill. Let the forum police themselves. It’s the way this is supposed to work. Just banning images site wide is pretty draconian and defeats the purpose of the fediverse. Blocking any images that could contain any level of nudity is also overkill. I’ll probably move to a self hosted server eventually.
I'm very happy if users who are comfortable with running live services set up their own instances, I think that's one of the best ways to ensure long-term success for Lemmy.
In response to "let the forum police themselves" - this is not a thing, unfortunately. While it's super important for lemm.ee that users downvote and report rule breaking content, somebody still has to deal with the consequences of these reports. Our admins are now already handling a three digit amount of reports daily. Additionally, there is a chance that illegal content is uploaded and never reported, but we still have a legal responsibility to deal with it.
Well, in that case, I think the Fediverse is in serious trouble. You will end up with too much fragmentation in how servers handle this sort of thing; it's definitely going to keep happening and probably get worse. I think delegating to the community of forum participants to handle the problem is in the spirit of the Fediverse. In either case, I admit it's up to each server owner to do what they feel is best. I suspect the Noster model of dumb repeaters is a better model.
That different instances handle issues differently is inevitable. Just from a legal standpoint they will HAVE to enforce different laws, depending on where they operate from.
However you have a point in the community helping out. I don't know what the application process is, but you might want to look into it. If you can just take over looking into some of these reports, it will help reduce this 3-digit load on admins.
If communities are never even given a view of the offending posts, they will obviously never be able to participate in the solution. I think communities that don’t address offending posts can and should be banned at the server level. Unless it is handled this way, then I guess server operators take complete ownership of the issue. I’m not even touching the topic of what constitutes an offending posts, which lends itself to all kinds of mis interpretations.
i’d kick your lazy a ass out if it were me, so be glad they aren’t me. they are volunteers working extremely hard so your azz can come here and make snide remarks about them at the worst possible time. ugh.
Whilst the discussion is all good and well, purely being here to gloat is pretty pathetic.
oh? and you know because they shared it with you? must be. how else would you know this? (aka, wtf is your problem? accusing people of crimes with zero evidence? it’s projection, my man. i suggest some time looking at yourself instead of accusing people of being pedos for modding this situation.