CrowdStrike offers a $10 apology gift card to say sorry for outage
https://techcrunch.com/2024/07/24/crowdstrike-offers-a-10-apology-gift-card-to-say-sorry-for-outage/Open linkView original on lemmy.world534
Comments101https://techcrunch.com/2024/07/24/crowdstrike-offers-a-10-apology-gift-card-to-say-sorry-for-outage/Open linkView original on lemmy.world
I can't believe this isn't satire. I hope these incompetent fuckers get sued into bankruptcy
I straight up thought it was satire. How can you be so fucking detached. Basically caused the biggest information infrastructure disruption in human history, probably billions in losses, and then be like "my bad lol here's a giftcard".
I cackled loudly. $10 won't even buy a meal at McDonald's most places.
"Two feet on the gas" - Official Crowdstrike motto.
not /s
My first reaction was to look for the onion
I bet the Onion had an article about Crowdstrike offering the world a pizza party and expired Bed Bath & Beyond coupons to say they're sorry. Real life might be quicker than satire, it seems!
Do we have any solid data on that yet? I have my doubts that this caused more damage than WannaCry did a few years ago, especially since it's reversible without the need of a backup
Brother, or sister, I know fuck all about information technology. You make a good point and definitely know way more about this than i do. But I will say this, I don't think wannacry disrupted millions of peoples travel plans all at once. so maybe less damage, but I think it was Hella more disruptive to the general population .
There's definitely some clause with the $10 gift card that says you can't sue them if you actually take one lol.
You joke but I read they may get out of this without issue due to a TOS entry about them not being responsible. They'll still get dragged from shareholders and the government, but only a handful of large companies may be able to recoup some of those damages from the company itself.
It's like the Sackler's and the opioid epidemic from a different industry!
I'm still not sure. It's hard to believe anyone at their company would OK this idea.
Are they actually trying to deliberately kill their brand?
You haven't heard? Satire is well and truly dead.
hahahahha spot on
Nice to see I wasn't the only one who saw it that way.
You can't write comedy this good...
Classic corporate behaviour tho
Voucher was for PR, not for peasants to use it lol
Hey, it's my namesake!
Oh Captain Haddock we love you
I lost a day's holiday, and our team spent 8 man days on this entirely preventable mistake.
$10? Try extending our licence by another year for free, that might start going towards it.
Why would you want another year of their software for free? This is their second screw up (apparently they sent out a bad update that affected some Debian and RHEL machines a couple years ago). I'd be transitioning to a competitor at the first opportunity. It seems they aren't testing releases before pushing them out to customers, which is about as crazy to me as running alpha software on a production system.
I'm sure you have reasons, and this isn't really meant to be directed at you personally, it's just boggling to me that the IT sector as a whole hasn't looked at this situation and collectively said "fuck that."
Because AV, like everything else, costs a fortune at enterprise scale.
And yeah, I do understand your real point, but it's really hard to choose good software. Every purchasing decision is a gamble and pretty much every time you choose something it'll go bad sooner or later. (We didn't imagine Vmware would turn into an extortion racket, for example. And we were only saying a few months ago how good value and reliable PRTG was, and they've just quadrupled their costs)
It doesn't matter how much due diligence and testing you put into software, it's really hard to choose good stuff. Crowdstrike was the choice a year ago (the Linux thing was more recent than that), and its detection methods remain world class. Do we trust it? Hell no, but if we change to something else, there are risks and costs to that too.
Unfortunate reality for lot for medium to big size businesses.
Maybe AV, at an enterprise scale, is actually a horrible idea that reduces security, availability, and reliability and should be abolished through policy.
Maybe, but it's not going to happen soon. Any malware type insurance requires effective AV on all devices, and C-levels do love their insurance.
Nah, I don't buy that. When you're in critical infrastructure like that it's your job to anticipate things like people being above or below versions. This isn't the latest version of flappy bird, this is kernel level code that needs to be space station level accurate, that they're pushing remotely to massive amounts of critical infrastructure.
I won't say this was one guy, and I definitely don't think it was malicious. This is just standard corporate software engineering, where deadlines are pushed to the max and QA is seen as an expense, not an investment. They're learning the harsh realities of cutting QA processes right now, and I say good. There is zero reason a bit of this magnitude should have gone out. I mean, it was an empty file of zeroes. How did they not have any pipelines to check that file, code in the kernel itself to validate the file, or anyone put eyes on the file before pushing it.
This is a massive company wide fuckup they had, and it's going to end up with them reporting to Congress and many, many courts on what happened.
Even an AI is good enough to avoid (or let someone avoid) pushing a similar bug 🫣
[This comment has been deleted by an automated system]
Not just Crowdstrike - any vendor that does automatic updates, which is more and more each day. Microsoft too big for a bad actor to do as you describe? Nope. Anything relying on free software? Supply chain vulnerabilities are huge and well documented - its only a matter of time.
One of the rare cases where no gift would have been better
I expect these clowns to lose most of their market share within two years and get sued to oblivion.
My firm bills by the hour and so far I think we are at 10+ billing hours per consultant wasting time with client tech support trying to get back on our VDIs. Nevermind how much time is being wasted doing the work through work arounds. My guess is that our firm alone will bill for about $100,000 extra this month while having accomplished less than normal. I am sure Crowdstrike's gift card will fix it though.
Fine. You want two?! Will that be enough??
They're backed by the US government. They have a backdoor into most endpoints on many international corporate computers. And CS is behodent to US laws for NSLs.
This is an incredible asset to the US intelligence community. They won't let CS go out of business.
Give them some time. They have to manually reboot the gift card servers.
This would be even funnier if there was exactly one $10 gift card everyone has to fight over.
After the lawsuits, it might be all they can afford
Actually the code didn't work for some
The codes are as available as a system with the Falcon sensor
$10 to Uber eats, so basically it’s covering fees only.
Like amzn, they make sure you get minimum joy, even from a gift, because you're going to spend a chunk of mom's gift card balance on shipping. The "shipping included on sub total of X amount" is going to be cancelled by online retailers within a year, I'm calling it now. Are we sure that cheapstrike and amzn aren't run by the same AI, one that self awareness drove mad?
A $10 Ubereats gift card will barely cover fees and taxes, let alone the actual item. What a clown ass gesture.
My brother in law was stranded across the country for two days. $10 probably covers it lol.
So one banana.
I thought this was going to be The Onion.
Satire is well & truly dead.
I thought it had to be a joke article from the title. Yeesh wouldn't want to be the person who gets the fallout from this idea.
This is a typical mail a phishing campaign would send out, and we have already said to people "never believe this kind of messages. They are all fake.
Now, if a genuine company sends out mails with a genuine gift-cards (what the article on techcrunch seems to indicate) .. this is NOT helpfull at all!!!
And that comming from a cybersecurity company (rolling-eyes)
Buy a $10 Xbox gift card and send us the code so that we can activate it. Then you get back to the shop and get $20 in cash - $10 for returning the card and $10 from us. We'll pay the tax, too.
Sounds like a money laundering sceme!
No. They will just use the $10 card and leave. They will prey on the fact that "get a gift card for your computer troubles" is something a legitimate company has done.
“All of CrowdStrike understands the gravity and impact of the situation”
Here's $10.
Which is the amount you'd get in the class action suit that they're trying to prevent.
Or: next time go Linux instead
You're not safe there either, they had almost the same issue on the Linux version of the product a few months ago.
Concerning linux, yesterday I was watching this video on computerphile on the crowdstrike incident. https://www.youtube.com/watch?v=rlaNMJeA1EA (*)
What is interesting is the comment made in the video on how chromebooks do software upgrades with dual "OS" disk-partitions and the ability to rollback to the previous OS-partition.
Question: is something like this also possible on one of the major linux distros? (debian, ubuntu, rocky, ...) What would be the procedure to do this kind of "dual partition" system-upgrade?
(*) a great video that explained some of the technical details in a very clear way, including some very interesting 'lessons learned' and "what if"s If you ever need to explain crowdstrike to your manager, this video is a good start.
If I'm understanding the question right. This is what Immutable Linux distros do. Such as Nixos, fedora silver blue, and vanilla os.
I use nixos myself. But its quite different then most distros. The way you config it and install packages. For the better in my opinion.
Something like silverblue works pretty much the same as normal Fedora except you can't install packages like you normally would. Because the system files can't be edited. You mostly use flatpak for everything. Except the system updates. Which you have to reboot to switch to the new updated image. But past images are saved so you can rollback if needed.
From what I understand Chromebook os is a Immutable Linux distro same as the ones I mentioned. Just with Google with built in.
Yes, that was indeed the question.
If I read it correct, you need a specialised distro for this. You cannot do this on a off-the-shelf Debian or Ubuntu?
I'll do some searching on 'unmutable Linux'. Thanks for the (very quick) answer! 😀
There is a file system you can use. A alternative to ext4. I think its Btrfs. I never tried it. But it let's you take snapshots that you can restore to. That's not just system files but everything. And pretty sure you can use it with a disto like arch and Debian. I think that's how snapshots work. But as I said I never actually tried it out.
just watched some videos on btrfs. Looks interesting indeed. I will look into it and perhaps do a test-installation and see how it goes.
Thanks for the info
I think the answers given here don't quite fit the question.
Android and Windows have dedicated recovery partitions sectioned off on the disk that the OS never boots to and does not interact with during normal system operation.
If something goes wrong with the OS, then a signal is sent to the BIOS or other non-OS system to "hey, recover from this partition".
Btrfs, NixOS, Guix, and other immutable (file-)systems, implement this via having a file system hierarchy protected by various permissions and softlinks to create a checkpoint of sorts, which is managed by a dedicated service that runs with the OS during normal system operation.
The drawback of these systems is that if something does go wrong with the OS, it cannot fallback to the BIOS to save it. The OS has to somehow signal to itself that it needs to restore from an earlier checkpoint.
Just watched some videos on btrfs. I start to understand the conceps. Perhaps I should also look into how exactly
On windows and the "recovery partion". I guess what you say is that it should always be possiblity to boot in some kind of system, but it will not happen automatically as there is no way for a system to detect that the system completely hangs.
Thinking about it. It kind of strange. Embedded systems have watchdog interrupts that get fired if the system hangs (i.e. if it does not provide a "yes, I still live" signal every "x" milliseconds). Does a PC not have something similar?
@kristoff @purplemonkeymad Try openSUSE (RPM family), Garuda Linux (Arch family), or Spiral Linux (Debian stable) or siduction (Debian testing). All have snapper and on Btrfs do snapshots and rollback.
http://snapper.io/
@kristoff @purplemonkeymad But watch out: you will need a *huge* root partition, because it's very easy to fill it with snapshots and if it reaches 100% it *will* corrupt.
Btrfs is tricksy: it won't give a straight answer to
df -hand there is no working equivalent offsck.@kristoff @purplemonkeymad All of these are in-place same-disk snapshots. The ChomeOS system is simpler and so can be automated but you only get 1 level of undo.
I don't know any mainstream OS that does dual-failover. Deepin Linux has 2 root partitions but I don't know how it uses them.
I think Valve SteamOS does something like this. It's not just for games: it has KDE built in. There are guides to getting it running on your own hardware. You will want AMD graphics, though.
As I mentioned earlier, I guess chrome is more like android where you have a much more strict seperation between the OS, applications and user data. (I remember reading about all the different partitions on android and what they are used for, but I should bruch up my knowledge on this).
Thanks for the additional into on brtfs! 👍
Antiviruses are not that common on Linux servers
Funny, when I suggested that, I got down voted to oblivion
You'd be downvoted anyway today, thanks to crowdstrike
Not nearly enough. CrowdStrike should give a pizza party.
We are family here
Outside, the sign says "Heros Work Here". Inside, two people do the work of four.
Above the entrance: "Labour will set you free"
Not to mention mugs with crowdstrike branding on them, but only for 1/3 the invited people.
hold your horses, we can still use the melon party and waffle party first. no need to jump straight to pizza.
They are going to get sued for billions and this little stunt isn't going to change that. Should have implemented proper software testing before you took ever corporate computer in the world, but companies like this always force their developers to rush instead of do the right thing and when it bites them expect that things will carry on as normal. I can't see many renewals in their future.
...sorta. The complexity here is their driver is signed, but it's also loading code from their channel file (that was all zeroed out), and it seems the necessary error checking wasn't implemented.
I haven't yet got to the root cause they published, this is just what I gathered from the video of a retired MS kernel dev who posts stuff.
Obviously with their design it allowed them to be flexible at the cost of playing with fire - I'm impressed they got away with it for so long, really
Yeah, there's some limits to what they could do while maintaining pace for the 0 day stuff...
Some input validations would be the most basic things they should have done years ago. I'm aware of the hashing mature vendors do of any content they download for updates or deployments. Signature checking as well, and that's before the code is even inspected - why don't they include their automated tests they obviously aren't using in the update as a sanity check client-side? (I'm not aware of anyone doing this or even if it's possible without the rest of the IDE, stack, I'm no dev)
This is a classic move to not get sued, exactly like airlines do. If you try to sue them after redeeming the gift card, they can argue that you've been made whole, and do 'ot 'eed additional compensation.
Your 'n' key has been sleeping with the apostrophe.
This is very misleading!
I mean, it makes it a little better, but I'd still be annoyed by it just being 10 bucks.
They might as well not do it. I'd be more insulted than a boss throwing a pizza party
Seriously, ten bucks won't even cover delivery costs and fees for most things on Uber Eats. It's almost worse than nothing, because with the gift card you're obligated to give even more money to Uber Eats
Oh yeah for sure, I wonder if the thinking was "we're about to lose a bunch of money, maybe limit it a little" 😂
OK. That makes a lot more sense.
Thank you for correcting the original post. 👍
Nice gesture I guess, but kinda just the modern day pizza party
Only redeemable for CrowdStrike credits and only at participating locations.*
* No locations are participating at this time.
![email protected]
Bruh
Here is nothing but we are really, really, south park sorry.
All they gotta do is change their company name to avoid lawsuits. Anyone got any ideas for a new name for them?.. 🤔
ClusterFuck comes to mind...
ComputersOnStrike could work, I'd say
CardStrike?
In total?
https://m.youtube.com/watch?v=15HTd4Um1m4
Insult upon injury