- Google Cloud’s US-East5-C (Columbus, Ohio) Zone had a six-hour power outage
- It was caused by a lack of unmatched power supply
- Over 20 services and some storage slices went down
Like the inept Titanic met its passing away as it sank, Google Cloud has recovered from a major interruption caused by an interruption of its uninterrupted power supply (UPS).
The company confirmed that its US-East5-C zone, otherwise known as Columbus, Ohio, experienced “deteriorating service or inaccessibility” for a period of six hours and 10 minutes on March 29, 2025, and accused it on a “loss of tool force in the affected zone.”
Over 20 Cloud services LED reduced performance or downtime as a result of the power cut, including Bigquery, Cloud SQL, Cloud VPN and Virtual Private Cloud.
Google’s uninterrupted power supply just had a pretty big failure
In its incident report, the company explained exactly what had happened: “This power outage triggered a cascading failure within the uninterrupted power supply system (UPS), which was responsible for maintaining power to the zone during such events.”
“The UPS system, which depends on batteries to bridge the gap between the power loss and the generator -power activation, experienced a critical battery failure,” the log continues.
Google’s Columbus Zone uses powerful chips from Intel Like Broadwell, Haswell, Skylake, Cascade Lake, Sapphire Rapids and Emerald Rapids as well as AMD EPYC ROM and MILAN processors to run its cloud computing services. The Cloud giant also noted that “a limited number of storage discs in the zone became unavailable during the power cut.”
Engineers were made aware of the power cut at. 12:54 pt on March 29, which successfully socialize with the failed UPS and recovered power via generator by 14:49 pt. Most services were brought back online fairly quickly, then, but some manual actions were required for a full restoration and thus six hours of power outage.
Google now promises to learn from this event, curing of cluster management and recovery paths and audit systems that do not automatically fail over, as well as working with its UPS supplier to mitigate future events.
