Microsoft has attributed last week’s global Microsoft 365 and Outlook outage to a software defect introduced in a July fix for its networking gateway servers. At the peak, more than 80 percent of those servers became unresponsive. Executive vice president Charles Lamanna said three factors combined: the latent defect, which caused servers to overwhelm under certain loads and was missed in testing focused on weaker hardware; a shift of Microsoft Teams traffic onto the primary gateway shared with Exchange Online about 15 hours earlier; and nearly 14 percent of gateway capacity already offline from an unrelated issue. Traffic management then redirected load, creating a cascading failure. Recovery took longer because restarted servers were immediately hit by the backlog. Exchange Online was hardest to restore. A final post-incident review is expected later this week.
Microsoft has attributed last week’s global Microsoft 365 and Outlook outage to a software defect introduced in a July fix for its networking gateway servers. At the peak, more than 80 percent of those servers became unresponsive. Executive vice president Charles Lamanna said three factors combined: the latent defect, which caused servers to overwhelm under certain loads and was missed in testing focused on weaker hardware; a shift of Microsoft Teams traffic onto the primary gateway shared with Exchange Online about 15 hours earlier; and nearly 14 percent of gateway capacity already offline from an unrelated issue. Traffic management then redirected load, creating a cascading failure. Recovery took longer because restarted servers were immediately hit by the backlog. Exchange Online was hardest to restore. A final post-incident review is expected later this week.