PhishER - Message Queue Delays and Intermittent Access Issues
Incident Report for KnowBe4
Postmortem

At approximately 3:40 p.m. (ET) on 06/29/20, the PhishER platform experienced an influx of message traffic within a short period of time, causing a delay in how quickly the messages could be processed. This delay inadvertently caused stress on some backend services, which caused the intermittent access issues some users experienced. PhishER experienced degraded console performance until around 5:04 p.m. (ET). Message queues were back to normal around 11:47 p.m. (ET). No data loss occurred as a result of this issue.

Posted Jun 30, 2020 - 10:16 EDT

Resolved
This issue has been resolved.
Posted Jun 30, 2020 - 07:22 EDT
Update
We are continuing to monitor for any further issues.
Posted Jun 29, 2020 - 22:27 EDT
Monitoring
A fix has been implemented and we are monitoring for any further issues.
Posted Jun 29, 2020 - 22:25 EDT
Identified
The cause of the issue has been identified and a fix is being implemented.
Posted Jun 29, 2020 - 17:53 EDT
Investigating
We are currently investigating reports that some message queues have been delayed and some users are experiencing intermittent console access issues.
Posted Jun 29, 2020 - 17:07 EDT
This incident affected: PhishER (Console).