Menu
Microsoft blames Aussie data centre outage on staff strength, failed automation

Microsoft blames Aussie data centre outage on staff strength, failed automation

The outage that occurred on August 30 led to downtime in Azure services pertaining to APIs, databases, and applications.

Credit: Dreamstime

Microsoft has blamed staff strength and failed automation for a data centre outage in Australia that took place on August 30, disabling users from accessing Azure, Microsoft 365, and Power Platform services for over 24 hours.

In a post-incident analysis report, Microsoft said the outage occurred due to a utility power sag in Australia’s East region, which in turn “tripped a subset of the cooling units offline in one data centre, within one of the Availability Zones.”

As the cooling units were not working properly, the rise in temperature forced an automated shutdown of the data centre in order to preserve data and infrastructure health, affecting compute, network, and storage services.

However, Microsoft said that the cooling units could have been restarted manually, which was not possible due to the unavailability of enough personnel at the data centre.

“Due to the size of the data centre campus, the staffing of the team at night was insufficient to restart the chillers in a timely manner. We have temporarily increased the team size from three to seven, until the underlying issues are better understood and appropriate mitigations can be put in place,” Microsoft wrote as part of the report.

In addition, the company said it is working on other major reforms, such as improving existing automation for the data centre to improve restoration of services when an incident occurs.

“We are exploring ways to improve existing automation to be more resilient to various voltage sag event types,” Microsoft said, adding that an evaluation was underway to ensure that the highest-load servers and their corresponding chillers restarted first.

In the past few months, Microsoft has reported several outages, especially the unavailability of M365 services. In July, an outage took out its OneDrive for Business and SharePoint Online services.

In June, users faced issues with Outlook Web, Teams, OneDrive for Business, and SharePoint for over eight hours. 

In May, the company reported that UK users were facing issues accessing some service offerings under Microsoft 365. In April, Microsoft said it was investigating an issue where certain users were unable to use the search functionality in multiple Microsoft 365 services. Outlook on the Web, Exchange Online, SharePoint Online, Microsoft Teams, and Outlook desktop clients were among the affected services.

In another incident in April, users could not access Microsoft 365 web applications, and Teams.

Microsoft also suffered a global outage in February, and yet again, its users could not access emails and Teams. It suffered a similar outage in January.


Follow Us

Join the newsletter!

Or

Sign up to gain exclusive access to email subscriptions, event invitations, competitions, giveaways, and much more.

Membership is free, and your security and privacy remain protected. View our privacy policy before signing up.

Error: Please check your email address.

Tags Microsoftdata centres

Events

EDGE 2024

Register your interest now for EDGE 2024!

Featured

Slideshows

How MSPs can capitalise on integrating AI into existing services

How MSPs can capitalise on integrating AI into existing services

​Given the pace of change, scale of digitalisation and evolution of generative AI, partners must get ahead of the trends to capture the best use of innovative AI solutions to develop new service opportunities. For MSPs, integrating AI capabilities into existing service portfolios can unlock enhancements in key areas including managed hosting, cloud computing and data centre management. This exclusive Reseller News roundtable in association with rhipe, a Crayon company and VMware, focused on how partners can integrate generative AI solutions into existing service offerings and unlocking new revenue streams.

How MSPs can capitalise on integrating AI into existing services
Access4 holds inaugural A/NZ Annual Conference

Access4 holds inaugural A/NZ Annual Conference

​Access4 held its inaugural Annual Conference in Port Douglass, Queensland, for Australia and New Zealand from 9-11 October, hosting partners from across the region with presentations on Access4 product updates, its 2023 Partner of the Year awards and more.

Access4 holds inaugural A/NZ Annual Conference
Show Comments