PhishER - Message Delays
Incident
Report for KnowBe4
Postmortem
On March 5, 2021 from approximately 1:45 to 5:30 p.m. (ET), processing of new messages, actions, and rule evaluations in PhishER experienced delays. These delays occurred because a batch of messages with a large quantity of links was sent to PhishER in a short period of time. The number of links caused a spike in queued tasks, temporarily blocking unrelated queue inquiries from processing.
These messages were moved into a low priority queue, allowing the query to process new inquiries at its normal rate. Moving the messages resolved the delays for all other users in PhishER. We are working on an enhancement to prevent this type of issue in the future.
No data loss occurred as a result of this issue.
Resolved
This issue has been resolved. Message queues have returned to normal.
Monitoring
We’ve implemented a fix for this issue and are monitoring the results. Some message delays might still be occurring but we are expecting queues to return to normal within two hours.
Identified
We’ve identified the cause of this issue and are implementing a fix.
Investigating
Some PhishER users may be experiencing message delays. We will update this page when we have more information.
This incident affected: PhishER (Notification Service).