Incident across several services
Sep 23, 10:11 UTCOngoing
Duration
Ongoing
Impact
Minor
Root cause
Not disclosed
GitHub, 90 days
68 incidents
Affected
API Requests
Final update
We are continuing to process the backlog of issue label updates for Projects. Label changes may still be delayed. All other services are operating normally.
Timeline
- Investigating · Sep 24, 03:14 UTC
We are continuing to process the backlog of issue label updates for Projects. Label changes may still be delayed. All other services are operating normally.
- Investigating · Sep 24, 02:00 UTC
We are continuing to process the backlog of issue label updates for Projects. Users may still see delays before label changes are reflected in Projects. All other services are operating normally.
- Investigating · Sep 24, 00:16 UTC
We've deployed a change intended to accelerate processing of the backlog of issue label updates in Projects. A sizable backlog still remains and we continue working through it. All other services are operating normally. We will provide another update within the next hour.
- Investigating · Sep 23, 21:39 UTC
Updates to issue labels may be delayed in being reflected in Projects. We are continuing to deploy a change that will accelerate processing of the backlog of label updates. All other services are available. We will provide another update within the next hour.
- Investigating · Sep 23, 20:26 UTC
We are preparing to deploy a change that will mitigate the impact.
- Investigating · Sep 23, 18:42 UTC
Continuing to investigate the lag that may be experienced in issue labels being accurately reflected in Projects. We are working on alternate solutions to process the backlog of label updates.
- Investigating · Sep 23, 17:30 UTC
We will post another update in approximately one hour to share our progress.
- Investigating · Sep 23, 17:01 UTC
Updates to issue labels may be delayed in being reflected in Projects by about ~10 minutes. We have added some capacity to work through the backlog more quickly, but it'll likely be a few hours to complete processing the full backlog of messages. All other services are available.
- Investigating · Sep 23, 13:35 UTC
Users may experience stale Project search results. We are working to increase indexing speed. All other services are available.
- Investigating · Sep 23, 11:38 UTC
We are seeing recovery for Projects. Users may experience stale search results for Projects while indexing catches up.
- Investigating · Sep 23, 10:58 UTC
The degradation affecting API Requests has been mitigated. We are monitoring to ensure stability.
- Investigating · Sep 23, 10:57 UTC
Database replicas have been restored. Org creation and the GitHub API are no longer degraded.
- Investigating · Sep 23, 10:20 UTC
Database replicas have detached. We're working to restore the database replicas. Users may experience issues beyond creating organizations and a degraded experience with the GitHub API and Projects.
- Investigating · Sep 23, 10:11 UTC
We are investigating reports of degraded performance for API Requests
More from GitHub
Full history| Started | Incident | Impact | Duration |
|---|---|---|---|
| Sep 2022:13 UTC | Incident with Pull Requests | minor | 1h 9m |
| Sep 1720:59 UTC | Elevated rate of errors for OpenAI models provided by Copilot | minor | 50m |
| Sep 1607:20 UTC | Degradation with Gemini 3.8 Flash | major | 10h 28m |
| Sep 1519:11 UTC | Disruption with some GitHub services | minor | 49m |
| Sep 1509:47 UTC | Disruption with some GitHub services | minor | 1h 30m |
| Sep 1418:40 UTC | Actions Larger Runner Jobs for some customers may be slow to start | minor | 55m |
From vendors' own status pages and disclosures. Times as reported. Logos via logo.dev; trademarks belong to their owners.