Asana downtime in all regions

Final update

We had approximately 1 hour of full downtime, where the Asana application was unavailable for most of our users. There was no data loss or security vulnerability. ‌ A routine configuration change affected the Asana application globally. While refactoring the configuration consumption, we missed an unintended side effect, causing the app to fail loading in all regions. We reverted the change and restarted servers for full recovery.

1. Postmortem · Nov 12, 17:15 UTC
   We had approximately 1 hour of full downtime, where the Asana application was unavailable for most of our users. There was no data loss or security vulnerability. ‌ A routine configuration change affected the Asana application globally. While refactoring the configuration consumption, we missed an unintended side effect, causing the app to fail loading in all regions. We reverted the change and restarted servers for full recovery.
2. Resolved · Nov 7, 22:08 UTC
   The incident is now resolved.
3. Monitoring · Nov 7, 22:03 UTC
   We are continuing to monitor for any further issues.
4. Monitoring · Nov 7, 21:59 UTC
   We are continuing to monitor for any further issues.
5. Monitoring · Nov 7, 21:56 UTC
   Functionality to the Asana webapp is now restored. We are still experiencing errors in the API, and are working to resolve them.
6. Identified · Nov 7, 21:39 UTC
   The root cause has been addressed, and some sessions are starting to see recovery. We are taking action to speed up recovery across all regions.
7. Identified · Nov 7, 21:24 UTC
   We are continuing to work on a fix for this issue.
8. Identified · Nov 7, 21:22 UTC
   We're currently experiencing a high volume of crashes for the Asana app. We've identified the cause and are actively working to resolve the issue. Thank you for your patience!