EU Region Error, Attachment, and Feedback latency
Jul 20, 11:11 UTCJul 20, 16:57 UTC
Duration
5h 46m
Impact
Critical
Root cause
Not disclosed
Sentry, 90 days
30 incidents
Affected
EU Ingestion - EU Attachment IngestionEU Ingestion - EU Error IngestionAlerting - EU Errors Alerting
Final update
We have fully burned the backlog of EU error events ingested during the incident. Latency is normal across all event types.
Timeline
- Resolved · Jul 20, 16:57 UTC
We have fully burned the backlog of EU error events ingested during the incident. Latency is normal across all event types.
- Monitoring · Jul 20, 16:09 UTC
We are still burning the backlog of EU region error events received during the incident. We are around halfway through the backlog.
- Monitoring · Jul 20, 14:50 UTC
The backlog on attachments in our EU region is fully burned, and latency is back to normal. While new error events are being processed with normal latency, we have a backlog of old error events that we are now burning through.
- Monitoring · Jul 20, 14:30 UTC
We have implemented a fix for our attachment ingestion issue in our EU region, and are monitoring while the backlog burns. We will update here when latency is back to normal.
- Identified · Jul 20, 13:47 UTC
We have implemented a fix for errors and feedback ingestion in our EU region, and latency for those pipelines has returned to normal. We are still working on fixing our attachments ingestion pipeline in EU.
- Identified · Jul 20, 13:19 UTC
We have identified the issue causing errors, attachments, and feedback events to have high ingest latency in our EU region and are continuing to work on a fix.
- Identified · Jul 20, 12:24 UTC
We have identified the issue causing errors, attachments, and feedback events to have high ingest latency in our EU region and are continuing to work on a fix.
- Identified · Jul 20, 12:10 UTC
We have identified the issue causing errors, attachments, and feedback events to have high ingest latency in our EU region and are continuing to work on a fix.
- Identified · Jul 20, 11:41 UTC
We have identified the issue causing errors, attachments, and feedback events to have high ingest latency in our EU region and are working on a fix.
- Investigating · Jul 20, 11:11 UTC
we are investigating an issue affecting errors ingestion in the DE/EU region
More from Sentry
Full history| Started | Incident | Impact | Duration |
|---|---|---|---|
| Sep 1810:17 UTC | Delays in spans ingestion | major | 41m |
| Sep 1808:36 UTC | Delayed ingestion of errors in EU | critical | 3h 20m |
| Sep 1016:13 UTC | Ingestion is delayed in US | minor | 57m |
| Sep 1015:25 UTC | Sentry.io elevated number of 500 errors | major | 1h 1m |
| Sep 114:16 UTC | Span ingestion is degraded in US | minor | 6h 14m |
| Sep 108:01 UTC | Sentry requests timing out / 504's | major | 3h 20m |
From vendors' own status pages and disclosures. Times as reported. Logos via logo.dev; trademarks belong to their owners.