AWS power outage with data loss

AWS power outage with data loss

  • September 8, 2019
Table of Contents

AWS power outage with data loss

On August 31st, 2019, an Amazon AWS US-EAST-1 datacenter in North Virginia experienced a power failure at 4:33 AM, which led to the datacenter’s backup generators to kick on. Unfortunately, these generators started failing at approximately 6:00 AM , which led to 7.5% of the EC2 instances and EBS volumes becoming unavailable. ‘1:30 PM PDT At 4:33 AM PDT one of ten data centers in one of the six Availability Zones in the US-EAST-1 Region saw a failure of utility power.

Our backup generators came online immediately but began failing at around 6:00 AM PDT. This impacted 7.5% of EC2 instances and EBS volumes in the Availability Zone. Power was fully restored to the impacted data center at 7:45 AM PDT.

By 10:45 AM PDT, all but 1% of instances had been recovered, and by 12:30 PM PDT only 0.5% of instances remained impaired. Since the beginning of the impact, we have been working to recover the remaining instances and volumes. A small number of remaining instances and volumes are hosted on hardware which was adversely affected by the loss of power.

We continue to work to recover all affected instances and volumes and will be communicating to the remaining impacted customers via the Personal Health Dashboard. For immediate recovery, we recommend replacing any remaining affected instances or volumes if possible.’

Source: bleepingcomputer.com

Tags :
Share :
comments powered by Disqus

Related Posts

A Technical Analysis of the Capital One Hack

A Technical Analysis of the Capital One Hack

The recent disclosure of yet another cloud security misconfiguration leading to the loss of sensitive personal information made the headlines this past week. This particular incident came with a bit more information from the indictment of the accused party, allowing us to piece together the revealed data and take an educated guess as to what may have transpired leading up to the loss of over 100 million credit card applications and 100 thousand social security numbers. At the root of the hack lies a common refrain: the misconfiguration of cloud infrastructure resources allowed an unauthorized user to elevate her privileges and compromise sensitive documents.

Read More
VPC Traffic Mirroring – Capture & Inspect Network Traffic

VPC Traffic Mirroring – Capture & Inspect Network Traffic

Running a complex network is not an easy job. In addition to simply keeping it up and running, you need to keep an ever-watchful eye out for unusual traffic patterns or content that could signify a network intrusion, a compromised instance, or some other anomaly. VPC Traffic Mirroring Today we are launching VPC Traffic Mirroring.

Read More
Toward a bastion-less world

Toward a bastion-less world

Using a bastion or jump server has been a common way to allow access to secure infrastructure in your virtual private cloud (VPC) and is integrated into several Quick Starts. Amazon Web Services (AWS) has recently released two new features that allow us to connect securely to private infrastructure without the need for a bastion host. This greatly improves your security and audit posture by centralizing access control and reducing inbound access.

Read More