Microsoft’s outage should terrify every enterprise

I’ve been in cloud computing long enough to see just about every kind of failure: bad deployments, network failures, identity outages, botched DNS changes, overloaded regions, misconfigured automation, and old-fashioned human error wrapped in a shiny cloud-native package. Most outages are expensive and annoying. Some are embarrassing. A few become case studies.

This Microsoft outage is different.

The scale, duration, and downstream impact of what just happened are unlike anything I’ve seen in my career. Microsoft 365, Outlook, Teams, Exchange Online, and Copilot were disrupted across more than 70 countries, with reports beginning November 12 and services not fully recovered until Saturday morning. That is not a temporary glitch. That is not a regional hiccup. That is an international cloud failure.

People can stand in a circle and kick Microsoft, and plenty will. But the more important question is what enterprises should learn from this. My clients are already asking the same thing: How do we reduce our dependency on this technology before the next failure costs us even more?

What Microsoft says happened

We are being told that this was not a cyberattack and not an AI-related incident. Microsoft pointed to a technical issue involving settings applied to servers after an update, along with a process for establishing secure connections that became overwhelmed and caused servers to stop handling requests. Other reports described servers failing automated health checks and becoming unavailable, which pushed more load onto the remaining infrastructure until those systems also became strained.

That explanation makes sense as far as it goes. A configuration or server update creates unexpected behavior. Automated health checks begin removing capacity. Load shifts elsewhere. Remaining systems become overwhelmed. Services fail. Engineers begin rolling things back and restoring systems gradually.

But that is the operational explanation. It is not the architectural explanation.

What likely happened

The likely story is much more troubling. A human-driven change hit a tightly coupled distributed system, and the system did exactly what fragile systems do under stress: cascaded. Health checks, routing, capacity management, secure connection handling, regional failover, identity, messaging queues, and service dependencies are all part of the architecture. When one part of that architecture starts making bad decisions at scale, redundancy can spread failure instead of containing it.

This should not be dismissed as a simple grouping of technologies that had a bad day. This was a cloud architecture failure. The cloud did not magically route around the problem. Automation did not save the business. A hyperscale platform did not absorb the blast radius. Architecture allowed the failure to propagate broadly enough to affect governments, healthcare systems, schools, courts, airlines, and businesses around the world.

This was caused by humans, and it can only be fixed by humans, but not with marketing statements, another dashboard, or another promise that the next post-incident review will result in better internal processes. It requires architectural changes, operational discipline, and a willingness to admit that cloud concentration has become a business risk.

This is Microsoft’s failure

Microsoft owns this one. That does not mean other cloud providers are immune. They are not. AWS, Google Cloud, Oracle, and every major SaaS provider all have their own risks and their own failure modes. But this specific outage belongs to Microsoft because Microsoft built, operated, changed, and recovered the platform that failed.

International outages of this degree and duration are unique. I have not seen anything like this in my career. Last year’s outages were expensive, but they were mostly annoying compared with this. This one stopped work. It stopped communication. It stopped shipping. It stopped business processes that people assumed would always be available.

That is the part executives need to understand. An outage like this is not just people being unable to send email for a few hours. Businesses were unable to produce, transact, schedule, deliver, reconcile, respond, or support customers. And when services return, the business does not simply restart like a light switch. Millions of people have to assess their systems, recover lost work, reconcile transactions, clear queues, and figure out what happened while the world was offline.

That recovery cost is rarely included in the outage headline.

The money is gone

Some estimates put lost productivity from the outage near $7 billion. I suspect the real economic impact, once downstream business interruption is fully understood, will be far larger. Whether billions or something approaching trillions across global dependency chains, the business reality is the same: Most of that money is gone. You are not getting it

[…]
Content was trimmed to protect the source. Please visit the original article for the full text.

This article has been indexed from InfoWorld