Incident summary
Some users were unable to open certain Gantt charts due to a bug that occurred after fixing a previous issue specific to large-scale environments.
Leadup
The incident occurred as a result of a bug fix aimed at large-scale environments. This fix inadvertently caused a new bug affecting access to Gantt charts.
Fault
The root cause was insufficient verification of the impact of the large-scale environment fix, which led to unanticipated errors in Gantt chart and template screens.
Impact
The incident affected all users of WBS Gantt-Chart for Jira, as some were unable to open specific Gantt charts.
Timeline (All times in UTC)
- September 2, 2024, 01:39 AM: Report received that Gantt list screen could not be displayed on a large-scale AC site.
- September 2, 2024, 01:40 AM: Investigation started.
- September 3, 2024, 01:14 PM: Bug with Gantt list screen resolved and deployed to production.
- September 4, 2024, 12:02 PM: Customer reported errors when accessing certain Gantt chart template screens.
- September 5, 2024, 02:11 AM: Determined that the issue was caused by the previous bug fix for the Gantt list screen.
- September 5, 2024, 02:12 AM: Began work on fixing the issue.
- September 5, 2024, 04:59 AM: Issue resolved and deployed to production.
- September 5, 2024, 05:46 AM: Confirmed that the app was functioning normally in the production environment and updated the incident status to "Resolved."
Root cause
The large-scale environment fix was not properly tested across all relevant areas, particularly for Gantt charts.
Testing should have covered both the Gantt list and the Gantt chart screens to ensure full functionality.
Corrective actions
- Implement more comprehensive testing procedures to ensure all areas affected by a fix are verified, not just the immediate area where the issue was found.
- Establish a testing environment that closely mirrors large-scale environments, including massive datasets, to prevent bugs specific to those conditions from being released.