Spyke
meta·Meta (lemm.ee)bysunaurus

2025-03-15 Unplanned lemm.ee downtime

Hey folks!

For the past few hours, lemm.ee has been bombarded with abnormal (almost definitely automated) traffic from a range of different IP addresses. This managed to overwhelm our servers, and we were offline for the past hour or so.

I was in the middle of celebrating my birthday, so response was a bit slow, but I believe we are recovering now, with mitigations in place to try and prevent further issues. Some of you may be inconvenienced by some bot checks when you browse lemm.ee, I am sorry about that, but it's necessary for now.

Sorry for the issues and I hope you have a nice weekend ahead!

View original on lemm.ee
251
Comments46
henfredemarsreply
infosec.pub

No way…

Frantically looks up the app.

Wow. Ah, Nice job. Thanks. I spend a lot of time using it every day.

12
henfredemarsreply
infosec.pub

Really, they’re 21 and managing such a large instance? Impressive. Is that true?

7
Dessalines
lemmy.ml

We've been dealing with these too at lemmy.ml, don't sweat it, we'll figure it out. Happy bday! Hope it was a good one.

14
Shadow
lemmy.ca

We saw a brief similar traffic spike approximately an hour ago, but it dropped off pretty quick. It seemed to focus on our mlmym front end and briefly overloaded our backends for a minute.

All the suspicious ips I've checked so far, are owned by chinese cloud providers (huawei, alibaba, tencent, etc)

11
Vyvanse
lemm.ee

Thank you for taking time out of your birthday to get lemm.ee back online. I hope the rest of your day goes great. Happy Birthday!

9
balssh
lemm.ee

Happy birthday! Thanks for your service and hope you enjoy your day.

9
[email protected]
sh.itjust.works

I don’t go to this school, but this seems like a particularly strange instance to attack. Were they scrapers? Which is also an incredibly strange thing to do. Spinning up an instance and directly pulling 6 years of content would be far easier.

7
Endmakerreply
ani.social

I read somewhere that the perpetuators are likely getting some practice in before attacking their actual targets. Lemmy instances just happened to be unlucky enough to be among the victims chosen for this rehearsal.

4
kratoz29
lemm.ee

Huh, I hope it wasn't me with my constant Summit (Lemmy client) refreshes (I was having issues with that app so I was doing a lot of API requests I guess) 🤣

Happy birthday! 🥳

7
revanite
lemm.ee

It seems this borked access for third party apps and the different front ends for Lemmy webpage. It is just showing network error

6
henfredemars
infosec.pub

Nice work! But remember, there are other instances, and as far as I know your users enjoy your services for free. Take it easy along the way.

6
revanitereply
lemm.ee

Same here. My android app and the front ends I use for the webpage can't connect to lemmee still.

4
alecbowles
lemm.ee

Happy Birthday! I’m using Voyager and experienced some issues and downtime as well. Sneaky bots.

4
_____
lemm.ee

I'm curious about steps you took in reaction to this, of course if sharing compromises the mitigation then by all means, you don't have to

and/or also how do you learn about managing attacks like these ?

4
dditty
lemm.ee

No problem, thanks as always for the hard work, and belated happy birthday!

1