Error ingestion degraded in US region
Jul 10, 22:43 UTCJul 11, 03:34 UTC
Duration
4h 51m
Impact
Critical
Root cause
Not disclosed
Sentry, 90 days
30 incidents
Affected
US Ingestion - US Error IngestionAlerting - US Errors Alerting
Final update
We have processed the backlog.
Timeline
- Resolved · Jul 11, 03:34 UTC
We have processed the backlog.
- Monitoring · Jul 11, 03:28 UTC
We have processed the majority of our backlog. We are now recovering the tail of the backlog.
- Identified · Jul 11, 02:37 UTC
We have processed the majority of our backlog. We are now recovering an additional 25M error events ranging from 22:20 to 23:20 UTC, which should complete in the next 2 hours
- Identified · Jul 11, 01:46 UTC
We expect recovery of historical data to be complete within the next 2 hours.
- Identified · Jul 11, 00:56 UTC
We are still working on recovery. New errors should be processed in real time, and we are working on recovering historical data.
- Identified · Jul 11, 00:04 UTC
We have identified the problem and are working on recovery. New errors are being processed in real time and we are in the process of consuming backlogged errors.
- Identified · Jul 10, 23:02 UTC
We have identified the problem and are working on a fix, ingested errors are delayed and will be processed once the fix is in place.
- Investigating · Jul 10, 22:43 UTC
We are currently investigating this issue.
More from Sentry
Full history| Started | Incident | Impact | Duration |
|---|---|---|---|
| Sep 1810:17 UTC | Delays in spans ingestion | major | 41m |
| Sep 1808:36 UTC | Delayed ingestion of errors in EU | critical | 3h 20m |
| Sep 1016:13 UTC | Ingestion is delayed in US | minor | 57m |
| Sep 1015:25 UTC | Sentry.io elevated number of 500 errors | major | 1h 1m |
| Sep 114:16 UTC | Span ingestion is degraded in US | minor | 6h 14m |
| Sep 108:01 UTC | Sentry requests timing out / 504's | major | 3h 20m |
From vendors' own status pages and disclosures. Times as reported. Logos via logo.dev; trademarks belong to their owners.