Event ingestion and retrieval delays for some North America customers
Final update
Between 07:41 AM PDT (UTC -07:00) and 12:37 PM PDT (UTC -07:00), some customers in North America experienced event ingestion and retrieval delays affecting Segments, Workflows, Timeline Events, and Report processing. This was caused by an underlying database infrastructure issue that increased read and write latency. As of 12:37 PM PDT (UTC -07:00), our affected tools are working properly and the incident has been fully resolved. HubSpot conducts a thorough review after each incident to understand the cause and prevent it from happening again. Learn more about HubSpot's commitment to reliability at www.HubSpot.com/reliability.
Timeline
- Resolved · Jul 28, 19:40 UTC
Between 07:41 AM PDT (UTC -07:00) and 12:37 PM PDT (UTC -07:00), some customers in North America experienced event ingestion and retrieval delays affecting Segments, Workflows, Timeline Events, and Report processing. This was caused by an underlying database infrastructure issue that increased read and write latency. As of 12:37 PM PDT (UTC -07:00), our affected tools are working properly and the incident has been fully resolved. HubSpot conducts a thorough review after each incident to understand the cause and prevent it from happening again. Learn more about HubSpot's commitment to reliability at www.HubSpot.com/reliability.
- Monitoring · Jul 28, 17:41 UTC
We’ve addressed the issue that caused event ingestion and retrieval delays for some customers in North America since 7:41 AM PDT (UTC -07:00). Real-time processing has resumed, and all queued events are now persisted and published. Some downstream tools, including Automation/Workflow, may take a little more time to fully reflect the recovered state. We’re continuing to monitor stability closely. The information on this page reflects our understanding of the incident and impact at the time of the update.
More from HubSpot
Full history| Started | Incident | Impact | Duration |
|---|---|---|---|
| Sep 2315:05 UTC | Some HubSpot tools may be unavailable for some users | none | 13m |
| Sep 816:21 UTC | Some HubSpot tools may be unavailable for some users | none | 59m |
| Sep 408:24 UTC | HubSpot is experiencing issues in the EU | none | 1h 20m |
| Aug 2109:57 UTC | Some Workflow actions may currently be delayed. | none | 2h 30m |
| Aug 1319:56 UTC | HubSpot is experiencing issues | none | 2h 14m |
| Aug 1119:31 UTC | Some HubSpot tools may be unavailable for some users | none | 14m |
Also caused by database
All| Started | Vendor | Incident | Impact | Duration |
|---|---|---|---|---|
| Sep 1309:16 UTC | Incident with several GitHub Services | critical | 1h 29m | |
| Sep 921:53 UTC | INC20000211 | critical | 1h 58m | |
| Aug 2616:17 UTC | Issues performance is degraded in the US | minor | 4h 19m | |
| Aug 1216:16 UTC | Incident with Pull Requests and Issues | minor | 25m | |
| Aug 818:57 UTC | Account Registration, Droplets, and Related Services | minor | 6h 3m | |
| Jul 2419:37 UTC | Incident with Pull Requests | critical | 46m |
From vendors' own status pages and disclosures. Times as reported. Logos via logo.dev; trademarks belong to their owners.