From approximately 4:00 p.m. Sunday, 04/28/2024, through 7:45 p.m. (UTC) Tuesday, 04/30/2024, the US instance of the KSAT console experienced issues with user provisioning.

This incident was caused by a large number of sync error reports. This congested the sync queue due to a recent backend change to how syncs are handled. To resolve this issue, we increased resources to process the queue backlog, and the user provisioning was able to return to normal performance by 7:45 p.m. (UTC) on Tuesday, 04/30/2024.

To prevent this type of issue in the future, we have changed the backend process to allow for greater control and quick adjustments when the queue experiences higher activity than normal. We will also be updating how user provisioning syncs are handled to increase efficiency and reduce the impact of unusually high sync error reporting activity.

No data loss occurred as a result of this issue.