At Least 750 US Hospitals Faced Disruptions During Last Year's CrowdStrike Outage, Study Finds

At least 759 US hospitals experienced network disruptions during the CrowdStrike outage on July 19, 2024, with more than 200 suffering outages that directly affected patient care services, according to a study published in JAMA Network Open by UC San Diego researchers. The researchers detected disruptions across 34% of the 2,232 hospital networks they scanned, finding outages in health records systems, fetal monitoring equipment, medical imaging storage, and patient transfer platforms.

Most services recovered within six hours, though some remained offline for more than 48 hours. CrowdStrike dismissed the study as "junk science," arguing the researchers failed to verify whether affected networks actually ran CrowdStrike software. The researchers defended their methodology, noting they could scan only about one-third of America's hospitals, suggesting the actual impact may have been significantly larger.

Fix was simple

[jhoegl]

Fix was simple, go into safe mode, run some dos commands to remove the update, boot up.

The problem was related to having to touch each server.
Running VMs, or servers with IPMI you could handle them remotely and quickly. if you have monitoring you know which servers got affected pretty quickly.
Took me an hour from recognizing the problem to fixing the few servers affected.
Luck and proper systems in place can reduce many outages.
I have to question those that took a long time, unless there are blocker

ESXi hosts where safe but not Hyper-V!

[Joe_Dragon]

ESXi hosts where safe but not Hyper-V!

Re:

[tlhIngan]

The servers were the easy part since they could be remotely managed - they likely were running some sort of VM hypervisor so it was a matter of booting a Windows recovery image, entering the Bitlocker key and then making the changes.

Of course, the key part was "Bitlocker key" since people probably didn't have it handy, and at least for servers it was likely in a state where you could copy and paste the key in so you weren't typing the number manually.

The hard part was the user aspect - repeating the same s

Crowdstrike

[registrations_suck]

Seems like Crowdstrike is the one doing the striking.

Maybe people should stop using that shitware.

Re: Crowdstrike

[Z00L00K]

Try to tell that to the management of the company I work for.

Re:Crowdstrike

[thegarbz]

Maybe people should stop using that shitware.

Cost or outages alone isn't a metric. The question is what is its reduction in incidents, and how much is the cost of not using them. Crowdstrike was nasty, but it's much cheaper than a successful ransomware attack.

Re:

[registrations_suck]

It's. It as if they're the only player in this market.

But you know why life hears of them? They fuck up. Not exactly a good reason to be known.

We don't think about hospitals

[rsilvergun]

As infrastructure. We think of them as businesses.

In the very near future that is going to bite us very very hard in the ass.

as it stands I am probably you too have lost jobs to people in countries that have universal Health Care like the United Kingdom because the high cost of not maintaining that infrastructure has increased the cost of hiring Americans substantially.

Re:

[Anonymous Coward]

We don't think about hospitals as infrastructure. We think of them as businesses.

Because they generally are? 87% of the hospitals in the US are nonfederal, aka. private hospitals running as nonprofits or for-profit.

And nobody learned...

[FritzTheCat1030]

Nobody learned and everyone is still running this crap. And McAfee/Trellix. My work laptop behaves like a machine from 2002 because all of the "security" software constantly eating resources. It would be expensive for companies to actually care about security, so they just all run the same garbage so when they get hacked they can just say..."Whaaat? It's not OUR fault. We were following 'industry standard best-practices'. Don't blame us".

Re:

[bill_mcgonigle]

Yep, I was working at an academic Medical Center in New Hampshire when they reassigned the Disaster Recovery guy to 1st level technical support to get him to quit.

The place became run by insurance and lawyers.

I quit when they said we were going to skip the code to prevent medication errors because it would be cheaper to settle the lawsuits.

Ok, but take the right lesson.

[SuiteSisterMary]

No organization should expect to be free from 'disruptions.'

Instead, they must have plans and processes in place to operate through 'disruptions.'