Flag delivery errors in Streaming API
Final update
This incident has been resolved. Between 12:00 and 12:12 PM PDT on August 26, server-side SDKs establishing new streaming connections experienced elevated error rates. Existing streaming connections were not affected, and polling requests continued to be served with error rates below 1%. Some requests to the LaunchDarkly web application also experienced elevated errors between approximately 11:29 AM and 12:03 PM PDT. The issue was caused by an internal configuration change, which has been reverted. All systems have recovered and are operating normally. No customer data was lost or compromised.
Timeline
- Resolved · Aug 26, 20:00 UTC
This incident has been resolved. Between 12:00 and 12:12 PM PDT on August 26, server-side SDKs establishing new streaming connections experienced elevated error rates. Existing streaming connections were not affected, and polling requests continued to be served with error rates below 1%. Some requests to the LaunchDarkly web application also experienced elevated errors between approximately 11:29 AM and 12:03 PM PDT. The issue was caused by an internal configuration change, which has been reverted. All systems have recovered and are operating normally. No customer data was lost or compromised.
- Monitoring · Aug 26, 19:45 UTC
Server-side streaming requests experienced elevated error rates between approximately 12:00 p.m. and 12:12 p.m. PDT, and some SDKs establishing new streaming connections may have been unable to connect. Polling request error rates remained below 1% throughout the incident. No customer data was lost or compromised. We apologize for the disruption.
More from LaunchDarkly
Full history| Started | Incident | Impact | Duration |
|---|---|---|---|
| Aug 2412:20 UTC | Email delivery failure affecting sign-up verification and notification emails | minor | 1h 15m |
| Aug 1710:36 UTC | Flag Updates via Polling API delayed | minor | 5h 26m |
| Aug 1414:47 UTC | Flag Updates delayed in US Region | minor | 2h |
| Aug 719:29 UTC | Elevated errors on server-side SDK streaming connections | minor | 46m |
| Jul 3117:44 UTC | Investigating - Known Impact | minor | 5h 10m |
| Jul 2819:24 UTC | Delay in Session Replay and Error ingest | minor | 41m |
Also caused by configuration change
All| Started | Vendor | Incident | Impact | Duration |
|---|---|---|---|---|
| Sep 2218:20 UTC | Phone Number APIs and Console Were Returning Incorrect 404 Responses | none | 0m | |
| Sep 1622:30 UTC | Hyperdrive Elevated Origin Connection Failure Rates | none | 0m | |
| Sep 1211:04 UTC | Some customers experiencing blurry image previews and download issues | minor | 8h 56m | |
| Sep 1107:18 UTC | INC20000213 | critical | 6h 1m | |
| Sep 407:34 UTC | INC20000199 | critical | 2h 36m | |
| Sep 109:58 UTC | INC20000190 | major | 3h 52m |
From vendors' own status pages and disclosures. Times as reported. Logos via logo.dev; trademarks belong to their owners.