About 8.5 million computers worldwide were taken out of commission by a bad software update over the weekend, grounding entire airports and downing non-emergency police and firefighting contact numbers as cash registers, banking systems and even social media servers failed one after the next.
Analysts estimate the system crash caused more than US$1 billion in damages including lost revenue and productivity, with at least A$200 million in damages in New South Wales alone, according to Business NSW.
The culprit was a routine sensor configuration update to major cybersecurity software provider CrowdStrike’s client systems, specifically those using the Falcon platform on Windows computers.
The update triggered a logic error resulting in what is called a Blue Screen of Death or BSOD (yes, that is a technical term), a system crash that disables the computer at the basic system operation level until it can be fixed.
Founder and CEO apologises
“I want to start with saying we're deeply sorry for the impact that we've caused to customers, to travellers, to anyone affected by this including our company," CrowdStrike CEO George Kurtz said in an appearance on US TV network NBC.
The outage is likely the largest cyber incident in history, eclipsing all previous hacks, glitches and outages.
The WannaCry cyberattack of 2017 is likely in second place for that dubious honour, having affected 300,000 computers in 150 countries with a ransomware attack that was estimated to have caused about US$4 billion in damages some five years after the event.
Microsoft (NASDAQ:MSFT) was quick to reassure users that the issue was not with Windows software, highlighting that less than 1% of computers using Windows were affected worldwide.
“It’s also a reminder of how important it is for all of us across the tech ecosystem to prioritise operating with safe deployment and disaster recovery using the mechanisms that exist,” Microsoft vice-president David Weston said.
Australian cybersecurity company StickmanCyber CEO Ajay Unni highlighted the irony of a cybersecurity software update causing so much damage.
“It’s a lesson to always update your software but obviously this is an extreme example,” Unni said.
“IT security tools are all designed to ensure that companies can continue to operate in the worst-case scenario of a data breach, so to be the root cause of a global IT outage is an unmitigated disaster.”
Fallout still unclear
Experts warn the outage is likely to draw attention from threat actors who view it as an opportunity – according to researchers at Secureworks, there has already been an increase in CrowdStrike related domain name registrations, which may indicate attempts to create scam websites.
CrowdStrike released a statement on Saturday explaining the company understood how the issue was created and that every effort was being made to identify the root cause.
“We are committed to identifying any foundational or workflow improvements that we can make to strengthen our process,” the statement read, “We will update our findings in the root cause analysis as the investigation progresses.”
There is no news about how the company might compensate those affected by the outage as yet, although CrowdStrike’s contract terms cap liability to fees paid, which may mean smaller companies and individuals are out of luck.
It remains to be seen what legal action larger entities like Microsoft might take, and whether CrowdStrike's contract terms are enforceable in this unusual circumstance.