Delays in search alerts
Incident Report for Papertrail
Resolved
We are back to normal! If you experience any issues, please reach out to us at support@papertrailapp.com.
Posted 2 months ago. Oct 04, 2018 - 17:01 PDT
Update
We are continuing to monitor for any further issues.
Posted 2 months ago. Oct 04, 2018 - 16:54 PDT
Update
We have absorbed the load spike and systems are back to normal. Delayed search alerts should be sent in the next 5 minutes.
Posted 2 months ago. Oct 04, 2018 - 16:54 PDT
Monitoring
Search queues, alerts, and website performance have returned to normal operation. SRE is continuing to monitor to ensure stability.
Posted 2 months ago. Oct 04, 2018 - 16:53 PDT
Update
After isolating the traffic spike, website performance has improved. Moving to degraded performance.
Posted 2 months ago. Oct 04, 2018 - 16:48 PDT
Identified
We have identified the issue as a load spike against our system, we are working to isolate the misbehaving traffic.
Posted 2 months ago. Oct 04, 2018 - 16:39 PDT
Update
We're seeing issues expand to more components of the serivce. Some users may experience HTTP 500 errors while interacting with Papertrail, including during live tail and log search. Search alerts may be delayed for some users. Provisioning Papertrail as a Heroku add-on may also fail for some users.

SRE is actively engaged and working on a solution.
Posted 2 months ago. Oct 04, 2018 - 16:28 PDT
Investigating
We are investigating an issue that might cause search alerts to be delayed, more info to come.
Posted 2 months ago. Oct 04, 2018 - 15:47 PDT
This incident affected: Website, Log Search, and Search Alerts.