Summary of Impact: Between 07:05 UTC and 09:45 UTC on 25 January 2023, customers experienced issues with networking connectivity, manifesting as network latency and/or timeouts when attempting to connect to Azure resources in Public Azure regions, as well as other Microsoft services including M365 and PowerBI.

Preliminary Root Cause: We determined that a change made to the Microsoft Wide Area Network (WAN) impacted connectivity between clients on the internet to Azure, as well as connectivity between services in different regions, as well as ExpressRoute connections.

Mitigation: We identified a recent change to WAN as the underlying cause and have rolled back this change. Networking telemetry shows recovery from 09:00 UTC onwards across all regions and services with the final networking equipment recovering at 09:35 UTC. Most impacted Microsoft services automatically recovered once network connectivity was restored, and we worked to recover the remaining impacted services.

Next Steps: We will follow up in 3 days with a preliminary Post Incident Report (PIR), which will cover the initial root cause and repair items. We'll follow that up 14 days later with a final PIR where we will share a deep dive into the incident. You can stay informed about Azure service issues, maintenance events, or advisories by creating custom service health alerts (https://aka.ms/ash-videos for video tutorials and https://aka.ms/ash-alerts for how-to documentation) and you will be notified via your preferred communication channel(s).

Summary of Impact: Between 7:08 UTC and 12:30 UTC on 25 Jan 2023, you were identified as a customer in Canada East, East US, South Central US, West US, and Canada Central who may have experienced latency or timeouts when deploying networking services through the Azure portal.

Preliminary Root Cause: We determined that our services were affected by a network latency brought about by a networking router that was taken out of rotation for maintenance during a spike in traffic. This led to increased congestion on some links.

Mitigation: The router was removed from service, and optimized the routes for the flow of traffic so services could resume normally.

Next Steps: A full root cause to investigate why this router caused wide spread impact will be developed. Stay informed about Azure service issues by creating custom service health alerts: https://aka.ms/ash-videos for video tutorials and https://aka.ms/ash-alerts for how-to documentation.