Handling spam accounts - best practices?
Hello NodeBB Community :wave:
I started a purge for our forum with the goal of deleting spam accounts.
Started with ~40000 user accounts and realized very fast that deleting them manually will be a Sisyphus task.
Vibe coded a usercleaner plugin with filters and stuff like that.
So with that filter, 70.20% of all accounts are spam. After that was cleaned out I looked deeper into more recent accounts and was a little shocked.
I identified a set of mail hosts used for verification over and over again. From the manage user view with 500 accounts did a little RegEx search with a browser plugin and got 381 of 500 users are from these mail hosts. So 76.2%.
After I complete my purge, from 39133 accounts 2754 are left. 93% of accounts where spam or the other way around only 7% "good" accounts.
note: will create some feature requests since I noticed some stuff in the manage user view
I found this plugin https://github.com/NodeBB-Community/nodebb-plugin-email-blacklist but also thought, that can't be it. These bots will just use other mail addresses. So, how do you guys handle this? Just ignore it?
Looking forward for some input from the community.
Thanks for reading and thinking about a clean forum with me :D
32 replies
Ive been following this thread and have been using a custom CRON script nightly to clean bots from my forums. I used Claude to generate an ALTCHA registration plugin here to hopefully help out some more, you can check it out here: https://github.com/sweetppro/nodebb-plugin-altcha :)
> @sweetp said: > > ve been following this thread and have been using a custom CRON script nightly to clean bots from my forums. I used Claude to generate an ALTCHA registration plugin here to hopefully help out some more, you can check it out here: > https://github.com/sweetppro/nodebb-plugin-altcha
wooow... :) Very good. I must test this ASAP Thank you @sweetp
@[email protected] cool! I've not heard of ALTCHA before. Nice to see additional entrants in the field.
On this forum we started using the registration queue with spam-be-gone plugin. It shows if the email/ip/username used during registration is found in the stopforumspam database.
Noticed some more things being outdated or bricked.
Translations not shown correctly. Report button shows
[[spam-be-gone:report-user]]and does apperently nothing because the click handler is only attached ontpl_url === 'admin/manage/registration'Will create some more issues and PRs :)
Got another idea to improve the spam-be-gone plugin - Check ASN of IP to get another spam indicator
GitHub issue #118
Since we already have the IP, the plugin could also check the IP address if it matches against knows ASNs. So if an IP is from a datacentre it is more likely to be spam. I am aware that this might also mark users as suspected spam if they use VPN/Proxy.
I already did an implementation but would like to discuss this first before creating a PR.
Demo Idea:
This was implemented using https://www.maxmind.com/en/home and https://github.com/X4BNet/lists_vpn There is a promising repo for ASN lookup, tho pretty new: https://github.com/openasn/openasn but good to have on the map for the future.
> @BrutalBirdie said: > > I am aware that this might also mark users as suspected spam if they use VPN/Proxy. > >
yep it's a problem For example, most of my users only connect via VPN
> @DownPW said: > > For example, most of my users only connect via VPN
Since this targets the registration an already existing user using a VPN is of no thought here. :thinking_face: Making it a config option for the plugin would resolve both needs, right? Example config: "Handle registration requests from known ASN as suspected spam" Maybe better wording for that config option? This way the NodeBB admin can configure the behaviour.
Edit.
here : https://github.com/BrutalBirdie/nodebb-plugin-usercleaner
I will test it...
Hey @downpw
> @DownPW said: > > You have wrote this plugin ?
Yes and No :sweat_smile: I did not write it, I vibe coded it. I would describe it as: I have thought it up and let AI do its thing.
Yea I have published it after I used it a little. Thought to myself, ehh, may be useful to someone else as well so why not put it on GitHub and npm.
Take a database backup before you let it loose on your codebase then :smile_cat:
> @[email protected] said: > > Take a database backup before you let it loose on your codebase then
I fully agree :laughing: Having a backup is always good and even more when playing with vibe code :grimacing: I will also add a disclaimer to the README of the repo, forgot it this time.
But in the spirit of true corperate IT:
> @BrutalBirdie said: > > Hey @downpw > > > @DownPW said: > > > > You have wrote this plugin ? > > Yes and No :sweat_smile: > I did not write it, I vibe coded it. > I would describe it as: I have thought it up and let AI do its thing. > > Yea I have published it after I used it a little. > Thought to myself, ehh, may be useful to someone else as well so why not put it on GitHub and npm.
Well, I assume you tested it a bit before releasing it publicly, especially given how sensitive it is.
Don't hesitate to keep it updated; I think quite a few people will find this plugin useful.
> @[email protected] said: > > Take a database backup before you let it loose on your codebase then > > >
Of course, I have daily backups of the database and the NodeBB folder ;)
By the way, I wonder what the best approach is when you need to restore a database backup to a server where the database already exists.
Delete the existing one and then restore the backup?
Same thing for the NodeBB folder?
I've never done this before; I'm just curious how MongoDB handles it.
I must test on VM
@[email protected] yes, because it's a full backup, it's cleanest to just nuke and pave.
That said I think
mongorestorehandles that for you. I don't think it goes through the trouble of merging data sets.> @DownPW said: > > Well, I assume you tested it a bit before releasing it publicly, especially given how sensitive it is.
I actually tested it on prod :sweat_smile: But since we have a solid backup/restore strategy and restoring is just 1-click :shrug:
> @DownPW said: > > Don't hesitate to keep it updated; I think quite a few people will find this plugin useful.
Will do and thanks.
> @DownPW said: > > By the way, I wonder what the best approach is when you need to restore a database backup to a server where the database already exists. > > Delete the existing one and then restore the backup? > > Same thing for the NodeBB folder? > > I've never done this before; I'm just curious how MongoDB handles it.
That is what we do. Throw everything away and rebuild from backup. In my opinion, a working full restore is always better to have comapred to a partial restore.
> @BrutalBirdie said: > > Vibe coded a usercleaner plugin with filters and stuff like that. > >
You have wrote this plugin ? I'm interrested :)
Hmmm I will have a look at that!
edit: Installed and set it up. Before we already had 20 new accounts from one of the spam mail hosts again. Will report back after some time to see how well it works :)
Yeah, registration queue is simple and effective, but the cost is a couple seconds of your time every day or so.
The queue also integrates with spam-be-gone plugin to provide additional signals for review (e.g. known spammer email domain or username, etc.)
We have a lot of old spam accounts on NodeBB too, those probably need to go. Lots of them sneak backlinks in their aboutme
> @[email protected] said: > > Lots of them sneak backlinks in their aboutme
That is why we put the aboutme and signature behind min. 5 rep to avoid that.
Check in. Again 10 accounts from
gsaserbuilder.xyzgot created. I have configured Spam Be Gone with Akismet, Project Honeypot, StopForumSpam and hCaptcha.I have mixed feelings about enabling the registration queue, but will try it anyway. edit: just set Registration Approval Type to Admin Approval and the link pointing to
$FQDN/admin/manage/registrationleads to aNot Found. Maybe need to restart the forum? Will try.> @BrutalBirdie said: > > Check in. > Again 10 accounts from gsaserbuilder.xyz got created. > I have configured Spam Be Gone with Akismet, Project Honeypot, StopForumSpam and hCaptcha.
Spam has been a long running bot overhead for sites for what seems like forever. I can only imagine AI has given this virtual dead army a diabolical boost.
In my simple mind I would have thought hCaptcha might provide a decent holdout against bots, am I too understand it had negligible effect even when combined with all the other mitigations? :worried:
> @omega said: > > In my simple mind I would have thought hCaptcha might provide a decent holdout against bots, am I too understand it had negligible effect even when combined with all the other mitigations?
In our case we still have a decent amount of spam accounts that try to get through. I have a suspicion how tho but I will need to investigate.
> BrutalBirdie said: > > edit: just set Registration Approval Type to Admin Approval and the link pointing to $FQDN/admin/manage/registration leads to a Not Found.
Found the real one, it is
/registration-queueand not/admin/manage/registration. So that is a "bug". https://github.com/NodeBB/NodeBB/pull/14825Another thought. There is the config option
Automatic Approval Time. So with spam-be-gone an option likeAutomatic Deny Time for suspected spamwould also be great. If my default action is to deny all suspected spam a setting like that would be good since if I forget to check the registration queue for the time configured for the Automatic Approval Time all the suspected spam account are approved.(:+1:/:-1:)?
Cross posting here, based on comment on this topic as there is similar thinking going on with maybe a novel idea jsut arrived at: https://community.nodebb.org/topic/19501/should-post-registration-queue-be-enabled-by-default/7
To avoid impacting sign-up friction and keep it as clean and smooth as possible, a bot-test pop-up as suggested here I believe would work a treat if it was post sign-up.
When sign-up completes successfully, throwing up an immediate checkpoint of some design for first time signups elegantly sidesteps making signup a PITA, creating a extra domain of protection where anything can be inserted to mitigate the dead army.
This solves the problem of loading up the signup stage with too much security in your face.
Caveat: Apologies if I'm way of the mark here and it already exists within the feature set of the post registration queue! :sweat:
Make the Data Centres go Brrrr...
How about a more advanced NodeBB community built hCaptcha, lets call its... NodeBBGame(r)Gate :wink.
The NodeBB peoples put their heads and hearts together and develop custom (nano) games that can be deployed as part of the sign up process. Totally and fully exclusive to the NodeBB platform.
Game hCaptcha has to present an higher computational overhead, so as to make them bot Data Centres go Brrrr... until they can Brrrr no more. :shrug:
The Human Approach
Why not ask the question, like - Are you registering using a VPN?
A) You filter the registration accordingly into vpn or non-vp groups, the later need admin approval etc.
B) You notify the registering user that there sign up will need manual approval
C) Your force or offer an option, to register with VPN off.
Speed Kills
I cna't remember are their speed of typing and other such checks handled by NodeBB?
My thinking with introducing a bit more human readable friction, is that bots would need to update to handle these fields if they ar not dynamically able to tackle, adding and removing periodically a question or two might stall waves.
In the age of the machine learning dead army maybe all bets are off, but instead of always defaulting to having tech solve our problems, why not use the solutions that already work and are tried and tested in a world we call home, the real world. :)
Even if Ai can be made parse and respond correctly, it may introduce a computational load that is unattractive.
Periodically adding or removing (here is where you could have a plugin that does this) or rotating all kinds of questions on the registration form might help:
There is a rather vast amount of question you can have pre-loaded that might route based on admin or plugin settings, I suppose this is a king of variation on the honeypot fields that only the bots are suppose to see and trip over, but I've found or my impression at least is, they do not work very well. You still get spam in contact form pages. Maybe less, but some still get through.
> @omega said: > > Even if Ai can be made parse and respond correctly, it may introduce a computational load that is unattractive.
I understand your thoughts but with an edge AI model that runs anywhere and uses little to none resources it simply analysis the HTML within seconds. No datacentre needed.
Not to mention the hurdles already added.
> @[email protected] said: > > Simplest form of spam prevention is still just a hidden input field, and then disqualify any registrants that fill that input with gibberish.
Is that not what project honeypot in this plugin is doing? :thinking_face: At least I would have expected it to do that.
You two @omega and @[email protected] just gave me some ideas.
If project honeypot from the spam-be-gone plugin does not add these honeypot input field, we should add them. But, we could add something else hidden. A set of hidden HTML label and input field added by the spam-be-gone plugin which attempts prompt injection. So even if an AI enhanced spambot figures out the registration process including all hurdles already added, maybe a prompt injection can add stop it or provide further details. :D Could work :shrug:
@BrutalBirdie I would imagine theoretically musing, there is no limit (other than your server resources) on how many pots and how much honey you could feed the Ai right?
So, like generative worlds, generative honey pots in registration forms. I'm no coder but powered by some infinity madelbrot algos if ya follow?
This may square the circle, if as @julian says it's still the most efficient method.
Make the data centres go Brrrr... (or Apple silicon!)
@BrutalBirdie
> @BrutalBirdie said: > > I understand your thoughts but with an edge AI model that runs anywhere and uses little to none resources it simply analysis the HTML within seconds. > No datacentre needed.
Aye but unless it's being used as a form of DDOS is there a rate limit set by the spam-admin (attacker), where the fail payload is just not worth the "oh bother!" anymore?
Thinking more on this, are we thinking too thinly in terms of threat profile, is there a component of hCaptcha type measures that may also be farmed out to people in a spam pipeline, so a hybrid model needs to be accounted for, if you take hCaptcha Images, they can be solved in theory by anyone who cna see and click a mouse, irrespective of language barriers, and that part may be farmed back to people, thus answering my own earlier query about hCaptcha effectiveness.
Then if so, to what degrees, and does this explain why some other measures still do not work as expected, because we have mis-profiled the threat.
Are we facing more empowered nuanced hybrid systems?
After-all what does the dead army do but given the single click being way more leverage beyond basic automation.
Simplest form of spam prevention is still just a hidden input field, and then disqualify any registrants that fill that input with gibberish.