Search Alerts are delayed

Incident Report for Papertrail

Resolved

Search Alerts have been returned to normal behavior.

Posted Feb 16, 2019 - 02:21 PST

Monitoring

Starting at 03:00 UTC on Feb 16th Search Alerts were delayed and we are continuing to work through the backlog.

Nearly all of the search alerts that would normally run between 03:00 UTC and 5:45 UTC were not run. A small number of search alerts that would normally run during that time searched a span of days rather than minutes or hours and may have resulted in alerts that were false positives.

The backlog of search alerts dating back to 5:45 UTC is being processed now.

Posted Feb 16, 2019 - 00:12 PST

Identified

We've identified the problem and we're working to get Search Alerts back to realtime.

Posted Feb 15, 2019 - 23:14 PST

Update

We continue to investigate.

Posted Feb 15, 2019 - 21:50 PST

Update

We are continuing to investigate and search alerts remain delayed.

Posted Feb 15, 2019 - 20:36 PST

Investigating

Search alerts are delayed.

Posted Feb 15, 2019 - 19:21 PST

This incident affected: Search Alerts.