QStash US Region: Schedule Degradation

Final update

# **Incident Postmortem: Scheduled Jobs Inconsistency in US Region** On May 1, 2026, we experienced an incident affecting a subset of schedules in the US region following a recent infrastructure update. The issue has been resolved, and all affected schedules have been restored. ## **Summary** As part of an ongoing scalability improvement, we recently updated scheduling infrastructure in the US region to a new architecture. During this transition, a legacy execution path remained in the codebase as a fallback mechanism. On May 1, a bug caused the system to revert to the legacy path. This resulted in inconsistent state between the old and new scheduling systems for some users. ## **Impact** The incident affected a limited number of users in the US region. **Most users were not affected**, and the vast majority of schedules continued operating normally throughout the incident. Users who did not update schedules during the transition window continued operating normally throughout the incident. **A subset of users who created, edited, paused, or deleted schedules between April 24 and May 1 may have experienced one or more of the following:** * Schedule updates not being reflected * Paus

1. Postmortem · May 6, 08:55 UTC
   # **Incident Postmortem: Scheduled Jobs Inconsistency in US Region** On May 1, 2026, we experienced an incident affecting a subset of schedules in the US region following a recent infrastructure update. The issue has been resolved, and all affected schedules have been restored. ## **Summary** As part of an ongoing scalability improvement, we recently updated scheduling infrastructure in the US region to a new architecture. During this transition, a legacy execution path remained in the codebase as a fallback mechanism. On May 1, a bug caused the system to revert to the legacy path. This resulted in inconsistent state between the old and new scheduling systems for some users. ## **Impact** The incident affected a limited number of users in the US region. **Most users were not affected**, and the vast majority of schedules continued operating normally throughout the incident. Users who did not update schedules during the transition window continued operating normally throughout the incident. **A subset of users who created, edited, paused, or deleted schedules between April 24 and May 1 may have experienced one or more of the following:** * Schedule updates not being reflected * Paused schedules becoming active again * Deleted schedules reappearing * Newly created schedules not executing as expected Schedules created after the transition may have stopped executing briefly before recovery. ## **Root Cause** During the transition, the new scheduling infrastructure became the sour
2. Resolved · May 6, 08:05 UTC
   This incident has been resolved.
3. Monitoring · May 5, 21:31 UTC
   Main schedule functionality is back to normal. We are currently checking if previously created schedules are delivered as expected before marking the incident as resolved.
4. Monitoring · May 5, 15:44 UTC
   A fix has been implemented and we are monitoring the results.
5. Identified · May 5, 14:29 UTC
   We are continuing to work on a fix for this issue.
6. Identified · May 5, 14:24 UTC
   We are currently experiencing issues in the US region. - Duplicate Deliveries: During this period, some scheduled jobs may be executed twice. - Schedule Disruption: Schedules created between April 24, 2026 and May 2, 2026 are currently not running. Our team is actively working on a fix. Once the migration is complete, affected schedules will resume normal operation. We will provide updates as progress continues.