On Tuesday, June 30, 2026, from approximately 07:40 to 19:15 (UTC), customers experienced incorrect results from PhishER's PhishML scoring. Affected emails received a PML:BYPASSED tag instead of a legitimate PhishML classification, and confidence scores were missing from impacted messages. Rules and actions that depend on PhishML results also did not activate.
This issue was caused by a code refactor introduced approximately two weeks earlier. This refactor introduced a faulty update that omitted essential drivers required for PhishML scoring to run. However, the issue remained dormant until another update triggered a new PhishML model deployment, which caused the scoring issue to emerge. To resolve this issue, our team rolled back to the last stable deployment and added more capacity to process the resulting backlog of email evaluations. PhishER's PhishML scoring returned to normal performance by 19:15 (UTC).
To prevent this type of issue in the future, we have improved health checks by introducing a new endpoint for smoke testing new models before deployment.