Elevated Packet Loss

Final update

Between 9:42 AM and 11:44 AM PDT, we experienced elevated network packet loss impacting AWS Direct Connect connectivity in the AP-SOUTH-1 Region. Our engineering team was automatically engaged at 9:46 AM to begin investigating. By 10:51 AM, we understood the root cause to be a configuration change made to a system responsible for assigning routes to devices. As we gained confidence in our mitigation steps, we parallelized our efforts to further reduce the packet loss. The issue is resolved and the service is operating normally.

1. Resolved · Jul 31, 18:54 UTC
   Between 9:42 AM and 11:44 AM PDT, we experienced elevated network packet loss impacting AWS Direct Connect connectivity in the AP-SOUTH-1 Region. Our engineering team was automatically engaged at 9:46 AM to begin investigating. By 10:51 AM, we understood the root cause to be a configuration change made to a system responsible for assigning routes to devices. As we gained confidence in our mitigation steps, we parallelized our efforts to further reduce the packet loss. The issue is resolved and the service is operating normally.
2. Update · Jul 31, 18:27 UTC
   We have identified the root cause to be related to a change made to a configuration system responsible for assigning routes to the devices. We have begun the work to reduce the packet loss that is impacting AWS Direct Connect in the AP-SOUTH-1 Region, and we expect recovery to take place gradually over the next 30 minutes. As we gain confidence in these efforts, we will seek to parallelize our efforts to speed recovery. We will provide another update by 12:00 PM PDT.
3. Update · Jul 31, 17:33 UTC
   We can confirm elevated network packet loss, impacting AWS Direct Connect connectivity in AP-SOUTH-1 Region. Our engineering team was automatically engaged at 9:54 AM to begin investigating the issue. There are no workarounds available at this time. We will provide another update by 11:30 AM PDT.