KMSAT, KCM, & PhishER - Intermittent Console Outages

Incident Report for KnowBe4

Postmortem

On Wednesday, 12/1/2021, from approximately 10:26 a.m. until Monday, 12/13/2021, at approximately 1:30 p.m. (ET), some users experienced intermittent connectivity issues and delays with various components of their KMSAT consoles.

These issues were caused by various underperforming backend components which were made apparent due to an influx of activity. After several upgrades and improvements, KMSAT returned to normal performance by 12/13/2021 at 1:30 p.m. (ET).

To prevent this type of issue in the future, we will release more updates to improve system performance in the coming weeks and months.

No data loss occurred as a result of this long-standing issue.

Posted Dec 14, 2021 - 18:17 UTC

Resolved

This incident has been resolved.

Posted Dec 14, 2021 - 18:12 UTC

Update

We are continuing to test further fixes and improvements designed to resolve the various issues users may still be experiencing with KMSAT. We will update this page as more information becomes available.

Posted Dec 13, 2021 - 16:28 UTC

Update

We are testing a number of fixes and improvements designed to resolve the various issues users may still be experiencing with KMSAT. We will update this page as more information becomes available.

Posted Dec 10, 2021 - 23:39 UTC

Update

We are continuing to work on a fix for this issue.

Posted Dec 09, 2021 - 16:00 UTC

Update

We are continuing to work on potential fixes to resolve the various issues users may still be experiencing with KMSAT. We will update this page as more information becomes available.

Posted Dec 09, 2021 - 15:57 UTC

Update

We are continuing to work on a fix for this issue.

Posted Dec 08, 2021 - 19:19 UTC

Identified

We are continuing to investigate reports of delays. We will update this page as more information becomes available.

Posted Dec 08, 2021 - 18:56 UTC

Monitoring

A fix has been implemented and we are monitoring the results.

Posted Dec 08, 2021 - 02:28 UTC

Update

We are continuing to work on a fix for this issue.

Posted Dec 07, 2021 - 19:44 UTC

Identified

We've identified the cause of the outages and various issues. We are working on implementing a fix and will continue to update this page with any new information or updates.

Posted Dec 07, 2021 - 19:28 UTC

Update

We have also received reports that users are experiencing issues with downloading files and notifications in KCM. We are investigating this issue as well and will update this page when we have more information.

Posted Dec 07, 2021 - 16:22 UTC

Update

We have also received reports that users are experiencing issues accessing their PhishER console. We are investigating this issue as well and will update this page when we have more information.

Posted Dec 07, 2021 - 16:07 UTC

Investigating

We have received reports that users are experiencing intermittent outages with their KMSAT console. We are investigating this issue and will update this page when we have more information.

Posted Dec 07, 2021 - 15:48 UTC

This incident affected: KCM GRC (Console), PhishER (Console, PhishRIP), and KnowBe4 Security Awareness Training (KSAT) (Console, Phishing, Training, Learner Experience (LX), Email Delivery, User Provisioning, Reporting).