Delays in search alerts
Incident Report for Papertrail
Resolved
We are back to normal! If you experience any issues, please reach out to us at support@papertrailapp.com.
Posted 11 days ago. Oct 04, 2018 - 17:01 PDT
Update
We are continuing to monitor for any further issues.
Posted 11 days ago. Oct 04, 2018 - 16:54 PDT
Update
We have absorbed the load spike and systems are back to normal. Delayed search alerts should be sent in the next 5 minutes.
Posted 11 days ago. Oct 04, 2018 - 16:54 PDT
Monitoring
Search queues, alerts, and website performance have returned to normal operation. SRE is continuing to monitor to ensure stability.
Posted 11 days ago. Oct 04, 2018 - 16:53 PDT
Update
After isolating the traffic spike, website performance has improved. Moving to degraded performance.
Posted 11 days ago. Oct 04, 2018 - 16:48 PDT
Identified
We have identified the issue as a load spike against our system, we are working to isolate the misbehaving traffic.
Posted 11 days ago. Oct 04, 2018 - 16:39 PDT
Update
We're seeing issues expand to more components of the serivce. Some users may experience HTTP 500 errors while interacting with Papertrail, including during live tail and log search. Search alerts may be delayed for some users. Provisioning Papertrail as a Heroku add-on may also fail for some users.

SRE is actively engaged and working on a solution.
Posted 11 days ago. Oct 04, 2018 - 16:28 PDT
Investigating
We are investigating an issue that might cause search alerts to be delayed, more info to come.
Posted 11 days ago. Oct 04, 2018 - 15:47 PDT
This incident affected: Website, Log Search, and Search Alerts.