News

Microsoft: Azure Outage Caused by Ubuntu Update Issue

According to Microsoft, Azure customers who had upgraded Canonical Ubuntu on Azure virtual machines may be running into domain name system (DNS) problems.

The problem specifically affected customers "running Ubuntu 18.04 (bionic) VMs [that were] recently upgraded to systemd version 237-3ubuntu10.54," Microsoft's alert indicated. Microsoft also confirmed the problem in this Azure Support Twitter post.

Canonical issued this announcement yesterday describing a "confirmed bug" for Ubuntu Systemd upgrades, where there are problems with resolving DNS. The announcement didn't mention the Azure virtual machines problems, though.

Microsoft's alert included a table showing that multiple Azure services were affected -- such as Azure Container Apps, Azure Database for PostgreSQL Azure Kubernetes Service and Azure VMware Solution. Presumably, those services were just affected for the Ubuntu upgraders using Systemd (see chart snippet).

[Click on image for larger view.] Figure 1. Azure services affected by a presumed Ubuntu 18.04 (bionic) upgrade problem on Aug. 30, 2022 at 6:00 UTC (source: snippet from Microsoft's "Azure Status" page).

The problems started at about 6:00 in the morning UTC time on Aug. 30, according to the alert, or about 11:00 p.m. Pacific Daylight Time on Aug. 29, per a timeanddate.com reckoning. The outages apparently have lasted 12 hours- plus, so far.

At press time, Microsoft was investigating the issue, which was just described as affecting Ubuntu 18.04 (bionic) upgraders using systemd. Here's Microsoft's statement to that effect and its advice:

This bug and a potential fix have been highlighted on the Canonical / Ubuntu site, which we encourage impacted customers to read: https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1988119
An additional potential workaround customers can consider is to reboot impacted VM instances so that they receive a fresh DHCP lease and new DNS resolver(s).
If you are running a VM with Ubuntu 18.04 image, and you are experiencing connectivity issues, we recommend you evaluate the above mitigation options.
If you are not experiencing impact on your Ubuntu 18.04 images, but you have unattended security updates enabled, we recommend you review this setting until the Ubuntu issue is mitigated.

Former Microsoft employee Kevin Beaumont described the DNS problems for Azure users in this Twitter post. He noted the odd circumstance that it's an Azure problem that's being blamed on Canonical.

"Azure are blaming Canonical (Ubuntu) but it only appears to impact Azure hosted VMs," Beaumont wrote.

The problem was also flagged by Dr. Nestori Syynimaa, a senior principal security researcher at Secureworks, in this Twitter post.

"These things happen all the time and will happen as long people are using technology," Syynimaa wrote in that post, adding "not on this scale that often though."

About the Author

Kurt Mackie is senior news producer for 1105 Media's Converge360 group.

Featured

  • Microsoft Appoints Althoff as New CEO for Commercial Business

    Microsoft CEO and chairman Satya Nadella on Wednesday announced the promotion of Judson Althoff to CEO of the company's commercial business, presenting the move as a response to the dramatic industrywide shifts caused by AI.

  • Broadcom Revamps VMware Partner Program Again

    Broadcom recently announced a significant update regarding its VMware Cloud Service Provider (VCSP) program, coinciding with the release of VMware Cloud Foundation (VCF) 9.0, a key component in Broadcom’s private cloud strategy.

  • Closeup of the new Copilot keyboard key

    Microsoft Updates Copilot To Add Context-Sensitive Agents to Teams, SharePoint

    Microsoft has rolled out a new public preview for collaborative "always on" agents in Microsoft 365 Copilot, bringing enhanced, context-aware tools into Teams channels, meetings, SharePoint sites, Planner workstreams and Viva Engage communities.

  • Windows 365 Cloud Apps Now Available for Public Preview

    Microsoft announced this week that Windows 365 Cloud Apps are now available for public preview. This aims to allow IT administrators to stream individual Windows applications from the cloud, removing the need to assign Cloud PCs to every user.