[Major] Some Users May See Errors Accessing The All Files Page

Final update

We recently addressed issues affecting availability of the Box All Files page. We would like to take the opportunity to further explain these issues and the steps we have taken to keep them from happening in the future. Between 5:42 AM PDT and 6:30 AM PDT on October 17th, some users may have experienced difficulties while working in Box. During this time, there was a degradation in availability of the All Files page. The issue occurred due to a bug in our code promotion pipeline, which promoted a disruptive change beyond the canary stage, causing wider impact. We were able to resolve the issue by rolling back the change. In addition, we are working to improve our release strategy and testing processes to prevent similar issues from occurring in the future.  ‌ **Analysis** The code that was promoted beyond the canary stage introduced an unintended change in the behavior of our feature flagging mechanism. The code change was intended to improve application performance by implementing a filtering-based approach. However, the feature flagging system was not yet fully compatible with this implementation, which resulted in a malfunction on the All Files page. ‌ **Corrective Actions** Bo

1. Postmortem · Oct 22, 19:47 UTC
   We recently addressed issues affecting availability of the Box All Files page. We would like to take the opportunity to further explain these issues and the steps we have taken to keep them from happening in the future. Between 5:42 AM PDT and 6:30 AM PDT on October 17th, some users may have experienced difficulties while working in Box. During this time, there was a degradation in availability of the All Files page. The issue occurred due to a bug in our code promotion pipeline, which promoted a disruptive change beyond the canary stage, causing wider impact. We were able to resolve the issue by rolling back the change. In addition, we are working to improve our release strategy and testing processes to prevent similar issues from occurring in the future.  ‌ **Analysis** The code that was promoted beyond the canary stage introduced an unintended change in the behavior of our feature flagging mechanism. The code change was intended to improve application performance by implementing a filtering-based approach. However, the feature flagging system was not yet fully compatible with this implementation, which resulted in a malfunction on the All Files page. ‌ **Corrective Actions** Box has initiated the following corrective actions: * Strengthened the canary deployment validation criteria in our release strategy. * Updated internal documentation to reflect feature-flag system dependencies and operational constraints. * Provided engineers clear guidance for safely integrating cha
2. Resolved · Oct 17, 15:07 UTC
   After further monitoring, this incident is now considered resolved. The All Files page service has been restored to full functionality. If you continue to experience any issues, please contact Box Support at https://support.box.com.
3. Monitoring · Oct 17, 14:37 UTC
   Our team has taken steps to remediate this issue and is seeing improvement for All Files experience. We are continuing to monitor for any additional impact.
4. Monitoring · Oct 17, 13:55 UTC
   Our team has taken steps to remediate this issue and is seeing improvement for All Files experience. We are continuing to monitor for any additional impact.
5. Identified · Oct 17, 13:36 UTC
   Our team is investigating an issue with Box All Files page. Users may see 500 HTTP errors while accessing the All Files page. We have identified the underlying cause of this issue and are working to take remediating steps. We will provide additional updates as they become available.