Massive internet outage reported: Google services, Cloudflare, Character.AI among dozens of services impacted
https://mashable.com/article/google-down-cloudflare-twitch-character-ai-internet-outageOpen linkView original on feddit.org696
Comments136https://mashable.com/article/google-down-cloudflare-twitch-character-ai-internet-outageOpen linkView original on feddit.org
Character.ai being mentioned in the same line is a sign of the times.
The people yearn for the digital anime waifus
If you're not running your waifu on a local machine she's just a digital prostitute.
The one downvote on this comment makes it like 20x funnier
When you stop using a search engine and use AI-generated responses for everything, I guess having your LLM backends go down could qualify as having a disruptive functional impact.
A disruption to disfunctional search engines does not sound bad.
Well, the actual headline is:
Character.ai is only mentioned in the lengthy list of services that are down.
It even swapped in Twitch when I read it at the time. Headlines be changing.
And when I went there, it said Google, Cloudflare, and Frank's Dildo Emporium all down, like of all the random sites, why pick FDE?
Oh shit, a fellow FDE-head in the wild... What do you think of the latest Bad Dragons they just got in? I got one for 50% off, and it was only slightly used by Frank.
Ai? Or a catch headline
It's a joke. I don't think there's really a website called Frank's Dildo Emporium. But if there is, lmk.
You could just google it-
oh wait
Still a very important service for many people who are alone.😢
Self host your desperation.
https://huggingface.co/Sao10K/L3-8B-Stheno-v3.2
What is character.ai? Have I been living under a stone? When I visit the page I have to log in just to learn what it is?
It's LLMs for ERP
... What's ERP? Or don't I want to know?
I put on my robe and wizard hat
Is that a bash reference? Jesus im old
Enterprise resource planning
Erotic resource planning
That's how I read it too at first- and as bad as 'erotic role play' with a LLM chatbot sounds, the other one is so much horribly worse.
My front-line guess is "Erotic Role Play" but I am not familiar with character.ai
I had guessed the nature of the site and "roleplay." It is a tad silly on my part to not have guessed "erotic"; thanks for getting me there (and not in an erotic way).
edit: Had left out the word "there."
Ohh that's why everything is slow and this is my sanity check.
But its scary how if Cloudflare went down most popular websites will go down with it
tell me more
It blocks anyone not using one of its preferred browsers, among other things. It's become the gatekeeper for a large fraction of the Internet.
It also flags my DNS filter and nearly every site I go to has to "check if I'm human"
What browsers does it block?
Minority browsers. Since I daily drive Pale Moon, I'm among the people affected. It's suspected that they test only the 3-4 most popular browsers, and whether anything else works with their code is up to luck.
You may think browsers with tiny market shares aren't important, but all new browsers start out that way. I fear for Ladybird if it ever makes it past the alpha stage, for instance.
I'm super interested in alternative browsers but never have the time to test them. I always wonder what the Internet would be like if we built it from scratch right now instead of having the legacy of 30 years of development to support.
He is not wrong. Look the whole internet is basically one centralized cloudflare if we continue this path.
https://arstechnica.com/security/2024/07/cloudflare-once-again-comes-under-pressure-for-enabling-abusive-sites/
great article, and I had no idea that happened to Brian Krebs, of all people! o.O
I do think the EFF makes a good point though, and I think personally I tend to be biased towards content neutrality over moderation (at least, more strongly the larger the platform is, and Cloudflare is very large). Not to the point of Xitter, obviously, but I think there's at least a reasonable argument for Cloudflare in this case.
that said, after some searching, I did find the following two articles, and I find their arguments against Cloudflare very compelling:
Fortunately I'm already using end-to-end SSL certs via Caddy, but now I'm considering just moving off Cloudflare entirely and instead providing regular backups to Internet Archive--most of the stuff I host is entirely static and very lightweight.
Like the bad kind
Redacted
You can try https://github.com/FlareSolverr/FlareSolverr
Interesting, I have a lot less problems with the Cloudflare verification, than with Google captchas. The slightest unusual thing seems to put you in captcha hell
When I get to that screen, I just x out most of the time. It's usually not that important.
Lemmy is working though. What else do we need?
I think that at least some instances use Cloudflare for various things, so depending upon what and how much stuff at Cloudflare is broken, some lemmy instances may be impacted.
I remember the day I went why are we all selfhosting and still relying on a site like Cloudflare?.
I'm glad I agreed with me on it.
this “me” character is so smart, aren’t they
Oh, he's the best. Keeps me out of trouble, that one.
I've migrated about half of my services off of cloudflare DNS proxy. Guess which half still worked this afternoon. (Self hosting pangolin with CrowdSec as replacement). I wasn't even using the cloudflare tunnel. Just their proxy for some bot mitigation.
been really enjoying pangolin so far! i got scared and ended up not installing crowdsec.
am i missing out? how hard would that be to add after the fact?
Can't speak to how difficult it would be to do after the fact. My guess is just adding in the lines in the docker compose. As for CrowdSec, it seems to run in the background and I haven't looked at it much. I've triggered it a few times and locked myself out. So I've added my home IP to the whitelist (it's running on a VPS). It also locked out my uptime Kuma so I whitelisted that too.
still hard. most projects still CDN online instead of packaging into a single application.
it's one of the most infuriating things about self-hosting. so I made my own self-hosted CDN and have rewrite rules that replace most public cdb domains and reroute them to my own local.
it would be great if I could just have something in the middle that would download once and cache locally on request, but it's effort that would be taken from my time afk.
By self-hosted you mean you have hardware geographically distributed? Like... boxes at friend's houses or...?
no, self-hosted meaning it's all on the local network. it's just a folder on NAS that has a local DNS route to it.
rewrite rules that rewrite the HTML and replaces "cdn.stuff.com" to "mycdn.local.net".
so not a true CDN, but a mock.
I use it for DNS and domain hosting, but that's it, none of their proxying stuff.
Why though? You have other options less detrimental to the internet than using and supporting them...
Domain registration and renewal is at cost, and DNS is free. I'm not sure what's detrimental to the internet about that...
My paid hosting is elsewhere, so they're not making any money off me.
For me it's because it's free, easy to use, and supported by ddclient.
I used Cloudflare tunneling for a while, then started to have similar thoughts. I'm off Cloudflare now.
This is my workday now
https://xkcd.com/303/
Nowadays they will tell you to work on something else in the meantime.
Your code can compile while you're in a retro, retrospectively looking back at the action items raised in a previous retro but not assigned any sprint time in the current sprint so you just raised a "what went wrong" about it not being planned and now you have more non-sprint work to do because every problem should be presented with a solution and if you could have that for the next retro they'll review this retro's action items and see if you have a proposal for fixing retro action items
This is why I call scrum masters scrumlords.
Thankfully my boss is hands off so I took a walk and then worked on an internal tooling side project that really only I will appreciate.
Scrumlords I love it
You have a good boss!
Quitting my last job for this one might be the best thing I've done since I tricked my wife into marrying down.
My mental and physical health has improved a ton.
As someone who hasn't worked anywhere with a profit-motive in over a decade, I don't know how you guys do it. Truly...
Yep
The problem was actually caused by Google Google Cloud went down hard on Thursday, and took Cloudflare and some of its customers with it.
I've been looking everywhere for somebody to explain it. Like, there are dozens of articles outlining exactly what happened. Which I don't need because I was there. I know what happened. I just have been wondering why and finally you have provided a link that actually gives me a reason.
So, thank you kind stranger.
I actually enjoy when this happens. All this centralized shit will not end well.
Centralized is control. Control is profit. It’s already not “well”, that’s why we’re here discussing it.
*Laughs in self-hosting services.
I wish I had that luxury with work.
All my personal stuff is 100% unaffected though!
You're not wrong. The gangly stacks of orchestrated black-box containers at a pubcloud beholden to shareholder whim is going to be a huge factor. Sorry grandma can't get her pills, but - phew - the number will still go up.
Same. Feeling pretty good about using Anubis instead of Cloudflare for our dinky systems.
Another reason to self host 😊
I’m pretty sure cloudflare has better uptime than you
This is dumb...
If you self host and your power goes out ...
Think about it
......My UPS kicks in.
Lol the power is out and this guy thinks he's gonna get overnight deliveries smh
/s
See, there's still a niche for the internal combustion engine over electric. Good luck driving your tesla in a blackout!
Maybe this guy lives in an area immune to random outages ? Never any power shutdown, internet disconnection or slowness. And his hardware never fails too 🫢
I prefer less uptime at saturday over a random rare outage.
I do agree , the web needs to be decentralized, but does your server have similar uptime to cloudflare.
Does it need to?
No but then the argument above falls flat, doesn't it?
At work just about all major services are super slow or down. It's like cloud strike again.
Oh well I still get paid. But it's crazy how much centralization affects the world wide web.
When The Cloud goes on Strike. Damn it they're just asking for more power, Scotty! Don't let those scabs cross the picket line!
Honestly, I didn't even notice.
We need to break up Cloudflare.
It sucks because up until the "sales team" rugpull, they're the cheapest (and closest to reality) for bandwidth cost, virtually all the other CDN providers charge astronomical prices and their margins are hundreds to thousands of percentage.
If the costs are mostly variable in how much they serve up, and uptime is sufficiently important, maybe have two CDNs and use the other one as a fallback when things start going tits-up?
Cloudflare tries to enforce pretty strong vendor lock in by requiring you use their nameservers.
Also subdelegate domains are an "enterprise" feature, so no luck there.
Basically the CDN market sucks, not a shocker Netflix, Google, Valve, and many others operate their own.
Hmm.
I'm not familiar with the constraint.
I assume that the way that this works is that I host content at www.foo.com and they have their nameserver resolve www.foo.com to different IPs based on the geolocation of the browsing user's IP.
Is it possible to convert www.foo.com to a CNAME that can be redirected away from their nameservers? Like, I make www.foo.com be a CNAME directed at www.foo-cloudflare-cdn.com. They own www.foo-cloudflare-cdn.com, they serve A or AAA queries there on their nameservers. But if I want fallback, I update the CNAME to point at www.foo-backup-cdn.com, which is served by a different CDN.
Are there technical barriers to that, do you know?
The way CDNs and virtual hosts work in general is to read the
hostfield in the HTTP header, otherwise unless you dedicate an IP for each domain / "web site" there would be no way to know what to serve.The issue is if you put the CNAME of foo
www.foo-cloudflare-cdn.com.then it will just resolve to whatever the A/AAAA record is for that, and send the host ofwww.foo.com– which they will only service if that domain is hosted with their nameservers (they run automated checks to make sure you're actually doing so). So there isn't really an easy way to just give cloudflare some subdomain, unless you pay them $$,$$$+ for the privilege.Valve actually does that, ironically enough, for the steam community web assets they use Fastly, Akamai, and CloudFront, all on subdomains of course 🙃.
But the point of CDNs is to direct connections to a geographically-near IP, yes?
The domain name that any CDN webserver in different regions will get in the HTTP request headers is going to be the same, CNAME or no.
Ah, okay, I could see someone having automated checks that actively prevent it.
That's generally right enough, the goal of a CDN is to deliver content from a server close to the consumer as possible (ideally on their ISP network using cache servers to avoid going out over the "wider internet".) – however CDN networks typically also use Anycast IP addresses, which means that all of the CDN servers across their network use the same pool of IP addresses, and BGP / the routing table dictate what actual physical server you get routed to. This is typically the ideal closest server, however sometimes you want certain IP pools in certain regions for legal (China), or technical reasons, so the IP address returned by a given A/AAAA lookup for a CDN isn't a given. There's also ECN and other optimization CDNs can do on the lookup side but that's outside of the scope here.
Yeah, so the CNAME just says "whatever A/AAAA address that resolves to" and the HTTP client will send whatever
HOSTit thinks its connecting to, meaning you can't "mask" the actual domain you're using by using a CNAME record.Technically if you have a totally static IP serving a single site, it's possible to ignore the HOST field and always serve that site, since logically, any request is only meant for that given site (this is basically the default site on something like Apache).
My main point is that there's really no getting around that CloudFlare requires you to be locked in to their platform even if you just wanna serve R2 files from a subdomain, and I personally find that a bit spooky, migrating nameservers can have very long propagation times leaving your site unreachable if they decide they don't want you as a customer anymore, or as a shakedown.
We just should not use cloudflare. Why is everybody still trying to use cloudflare.
It drives me nuts. The default answer on all network problems: why don't you just use cloudflare? No! No no.
Counterpoint: I find networking challenging, and I'm not particularly accountable for the natural consequences of not knowing how stuff works.
This could end fine for everyone? Maybe an AI will understand it for me. (This is sarcasm. I agree with you.)
Dafuq is a character.ai and why is it in the same list as google and cloudflare? Like yea my local grocery store had some issues should that be on the list too?
An AI chatbot that pretends to be a character and you roleplay with it.
Back in my day, we just told Cleverbot to suck our dicks
Had to include something AI because of the craze, probably picked at random
It is very used actually. There's a lot of lonely people... :(
(I don't use it tho, I just yell at the void on Fedi)
c/selfhosted having a field day rn
Yeah, wonder how those cloudflare(d) tunnels werd keeping up.
The answer always seems to be: Cloudflare is fucking up again. This isn't the first time.
Google Cloud status page:
https://status.cloud.google.com/
Thanks I have updated the Google link to this instead of downdector.
Holy shit, goddamn near everything is down, nearly everywhere.
Fucking Tracer Tong ending from Deus Ex, hope somebody has a save file they can reload from soon, otherwise... yeah...
Was trying to order food online and that was apparently affected.
Gotta love having all our eggs in one basket, eh? We learned something from the big ass east coast AWS outage!
Uh huh, just before a major democracy attacked an evil, evil country. Must be coincidence.
Everything is a conspiracy when you don't know how anything works
So tell me, how does the world work? We all hold hands and hug and cry together?
I'm gonna Kum ba yah so hard, you won't be able to stop me!
Maybe I don't want you to stop, big boy.
eventually
You're funny.
And that other huge democracy ongoing attacks on its own cities and economy.
I think https://pkg.go.dev/ is down too
npm is having issues as well with publish and installs. Fun fun day.
Time to forget about code and relax
The whole internet is down. Unless you self host
Stuff hosted by cloudflare is not the whole internet
Almost all services on the internet are behind cloudflare ddos protection.
??? My shit is working.
Strangely enough, twitch streams and chat were working fine but everything else was broken, had an empty following list.
It's seen tech talks by Twitch's engineering team. Some of those folks are scary smart.
Not that it takes that much cleverness to avoid using CloudFlare, of course.
But might be related. Twitch had some clever fallbacks and work-arounds for slow Internet, in that tech talk.
Good, we have a reason to touch grass now
Oh no, not Cloudflare!
Oh right I use NextDNS, so anyway...
Oh, its also Google, Twitch, Discord and Etsy... and some AI girlfriend generator... and Spotify?
... People still use those? Aren't they worried about all the security and privacy concerns, the exploitative business models?
No?
Huh. Well, dang, sucks for those users I guess.
Ahem, annnyyyywaaaayyyy...
...
Ok, serious mode / tinfoil hat:
... Did maybe someone in the Trump admin just try to do a US internet blackout, given all the insane shit happening, and pulled a few wrong cords out?
...
Either way... and I cannot believe I am citing Ready Player One as a positive example of anything, but such is the nature of our collective degeneracy:
Tuesdays and Thursdays are now no internet days.
Go outside, touch grass, realize you're addicted to social media like a drug, and begin detoxing.
...
EDIT: Holy shit, Israel just struck Iran at multiple locations, presumably all or many related to their nuclear program... including Tehran.
https://www.nytimes.com/live/2025/06/12/world/israel-iran-us-nuclear
https://www.axios.com/2025/06/13/israel-strike-iran-trump-nuclear-talks
https://www.nbcnews.com/world/iran/live-blog/israel-strikes-iran-middle-east-conflict-live-updates-rcna212727
https://www.wsj.com/livecoverage/israel-iran-strike-conflict
info blackout tinfoilhattery intensifies
Owhooo your virtual girl friend is offline!
Jokes on you, my virtual girlfriend runs locally!
No, she's just ghosting you.
Lemmy included.
Speaking as someone who managed to quit smoking, but still uses a vape, who is focusing on doing daily PT after being maimed a while back...
A step in the right direction is a step in the right direction, and taking one step at a time is better than none.
Lemmy ain't corporate like Reddit, it ain't parasocial, only a very small number of users actually directly link their real identity to their psuedonym, and it doesn't have a worldclass datamining/profiling algo perfecting what content to feed to specifically you, to make you angry or insecure enough to keep you engaged.
Perfect?
No.
Leagues better than every alternative?
Far as I can tell, yes.
Fundamentally, the brain still receives "bite-size-chocolate" dopamine hits from Lemmy by receiving positive affirmations from upvotes, cortisol from downvotes, and lends to dark behavior patterns like any forum. Adrenaline dumps when engaged in "online arguments".
The nature of both anonymous or personally identifiable online forums' neurochemical stimulation of our over-sized and over-active (but evolutionarily necessary for our ancestors' survival) amygdala fosters social media addiction.
People like Lemmy for many reasons. Some of them are good. However, let's not pretend that it's "all of the good with none of the bad." It's healthy to be skeptical of Lemmy instances too. Screen time is the enemy.
Yeah, a great part of Lemmy's fundamental design is that it gives the user so much ability to block specific toxic users and communities and even entire instances from being seen by that user.
A user who is interested in self regulating or limiting that potential adrenaline overload ... is aided by Lemmy in doing so.
This is significantly different from how its nearest equivalent, reddit, operated untill about 2 years ago, when they finally added an actual block user ability.
Still don't think you can block the entire user group of subreddit communities, the way you can block an an entire Lemmy instance, if you want to.
(At least not without some third party script or software... which are probably all broken by now given how hard reddit is cracking down on its API?)
Also, moderation and admin logs are significantly less opague than on reddit.
To the best of my knowledge, on lemmy, you can't admin edit the post of someone you are arguing with to frame them, basically, and then turn them into a strawman of themselves, and then win that argument with them, and then ban them... as has happened on reddit.
Also Also, ... lemmy at least not yet does not appear to have a problem with a massive flood of ai bots posting god knows what % of the actual content.
...
Not saying Lemmy is perfect.
I'm saying its better.
And I guess I'm also saying there's a difference between being an alcoholic and enjoying an occasional drink from time to time.
Generally: Yes, of course, approach any online messsge board or social media with caution and skepticism... but different platforms can be significantly more conducive to generating negative mental health outcomes than others...
... short form video platforms collapse your attention span, anything that allows advertisements or 'influencers' who are basically just walking talking brand ambassadors lie to you to sell you all kinds of bs...
Thats not present on lemmy, at least not that I've seen... so in those ways, lemmy is the marijuana to say Tiktok's fentanyl.