Delays starting jobs on Docker (Gen 2) resource classes
Final update
On July 13 between 15:30 UTC and 22:40 UTC, we experienced delays starting jobs on Docker (Gen 2) resource classes due to capacity constraints from our cloud provider. The same issue recurred on July 14 between 15:45 UTC and 22:50 UTC. Delays reached up to 25 minutes on July 13 and up to 24 minutes on July 14. The medium+ and 2 X-large+ resource classes saw the longest waits. Earlier updates in this incident reported figures based on average wait times, which didn't reflect the maximum impact some customers may have experienced. Wait times have now returned to normal. We are continuing to work with our cloud provider to add capacity ahead of the next peak period. Thank you for your patience.
Timeline
- Resolved · Jul 15, 00:32 UTC
On July 13 between 15:30 UTC and 22:40 UTC, we experienced delays starting jobs on Docker (Gen 2) resource classes due to capacity constraints from our cloud provider. The same issue recurred on July 14 between 15:45 UTC and 22:50 UTC. Delays reached up to 25 minutes on July 13 and up to 24 minutes on July 14. The medium+ and 2 X-large+ resource classes saw the longest waits. Earlier updates in this incident reported figures based on average wait times, which didn't reflect the maximum impact some customers may have experienced. Wait times have now returned to normal. We are continuing to work with our cloud provider to add capacity ahead of the next peak period. Thank you for your patience.
- Identified · Jul 14, 22:54 UTC
On July 13 between 15:30 UTC and 22:40 UTC, we experienced delays starting jobs on Docker (Gen 2) resource classes due to capacity constraints from our cloud provider. Delays reached up to approximately 1 minute 48 seconds. We are continuing to experience these delays again since approximately 15:45 UTC. So far, delays have reached up to approximately 1 minute 19 seconds. We are working with our cloud provider to add capacity and will update this incident as the situation changes.
More from CircleCI
Full history| Started | Incident | Impact | Duration |
|---|---|---|---|
| Sep 2318:13 UTC | Delays in processing new signups and plan changes | none | 1h 53m |
| Sep 2116:22 UTC | Delays starting Linux Machine jobs | minor | 47m |
| Sep 1614:55 UTC | Customers may experience delays in UI updates | minor | 4h 35m |
| Sep 1514:42 UTC | Customers may experience delays in UI updates | minor | 2h 20m |
| Sep 1411:59 UTC | Increased wait times for macOS jobs | minor | 1h |
| Sep 1020:25 UTC | Issues loading pipelines for a small number of projects | minor | 1m |
Also caused by third-party dependency
All| Started | Vendor | Incident | Impact | Duration |
|---|---|---|---|---|
| Sep 1609:48 UTC | Support Portal Service Disruption | minor | 11h 21m | |
| Sep 1608:31 UTC | Support Helpdesk Availability Issues | minor | 6h 27m | |
| Sep 417:03 UTC | Degradation of Hosted Grafana in US Central Region | major | 3h 39m | |
| Sep 416:05 UTC | Elevated Linux worker queue times | major | 15h 4m | |
| Sep 115:25 UTC | Investigating issues in US Central (prod-us-central-0, prod-us-central-5) | minor | 4h 54m | |
| Aug 3106:26 UTC | Packet loss in ORD | minor | 1h 44m |
From vendors' own status pages and disclosures. Times as reported. Logos via logo.dev; trademarks belong to their owners.