News

UPDATED: Windows Azure Storage Restored After Worldwide Outage

• By Jeffrey Schwartz
• February 25, 2013

Service was restored to Microsoft's Windows Azure storage on Saturday, after an expired SSL certificate left customers unable to access their data beginning Friday afternoon.

The SSL certificate expired on Friday afternoon and remained unavailable until Saturday morning. The worldwide outage affected HTTPS traffic accessing storage, though did not impact less secure HTTP traffic, the company confirmed on the Windows Azure Dashboard.

"We have executed repair steps to update SSL certificate on the impacted clusters and have recovered to over 99 percent availability across all sub-regions," according to an updated alert on the dashboard Saturday. "We will continue monitoring the health of the Storage service and SSL traffic for the next 24 hrs. Customers may experience intermittent failures during this period. We apologize for any inconvenience this causes our customers."

That did little to console some customers. "Most of our apps are screwed up now! "WHATS NEXT? All compute instances die because someone at the data center switched them off?," wrote one customer on a Windows Azure MSDN forum.

"This is unacceptable, I'm supposed to release an enterprise app on this platform?," added Microsoft partner MJ Fara. "Imagine how many phone calls I would have gotten by now from very angry customers. Sad."

And another partner who identified himself as Matt from Cambridge said: "Pretty amazing that the entire Windows Azure storage platform is offline globally."

Others took it in stride, noting it was an error any customer or partner could make. "I bet a lot of people here have accidentally let an SSL cert expire, or nearly done so," said Brian Reischl of Sepia Labs, a company consisting of former NewsGator employees, who first pointed to the SSL certificate with a screen capture Friday. "I know I have. It's easy to forget, right? It's an amateur mistake, but it happens. You end up with some egg on your face, add a calendar reminder for next year, and move on."

The original story follows:

Microsoft confirmed on Friday afternoon that a worldwide outage has hit Microsoft's Windows Azure cloud service, specifically its storage services.

The cause appears to be from an expired SSL certificate, which Microsoft is not confirming at this time. An alert on the company's Windows Azure dashboard reports that the worldwide outage is affecting SSL traffic, thereby making its storage services unavailable.

"We have identified the root cause and are validating the recovery options before implementing them," according to the alert. "Further updates will be published to keep you apprised of the situation. We apologize for any inconvenience this causes our customers."

A company spokeswoman said Microsoft became aware of the issue at around 8:45 p.m. UTC (United Kingdom time). "We are actively investigating this issue and working to resolve it as soon as possible," the spokeswoman said in an e-mail. "Updates will be published to the Windows Azure dashboard to keep customers apprised of the situation."

Brian Reischl of Sepia Labs, a company consisting of former NewsGator employees, pointed to the SSL certificate with a screen capture, which shows today's expiration date.

"So is it just me, or did the HTTPS certificate for Azure Storage just expire?," Reischl posted on the Windows Azure forum.

"If your app is set up to accept reconfiguration on the fly, you might be able to flip to using HTTP instead of HTTPS," he advised. "If not, you're pretty much up a creek. Doing new deployments seems to use storage under the covers, so you can't right now."

A flurry of posts hit Twitter regarding the outage and the expired certificate.

"Windows Azure Storage, Amateur Hour Outage," tweeted Greg Knieriemen, a storage, cloud and virtualization evangelist for Hitachi Data Systems.

Charlie Maitland, a consultant in the U.K. specializing in the Microsoft stack, lamented: "Running a global IT business -- $37b: Building a cloud data system -- $5b: Renewing a $10 SSL cert -- Priceless."

Microsoft is advising its customers to monitor the Windows Azure dashboard for further updates on the current outage. The service has caused periodic troubles for some customers, with notable outages occurring in February 2012 and December 2012.