Papertrail
All Systems Operational
Website   Operational
Log Collection   Operational
Log Search   Operational
Search Alerts   Operational
Operational
Degraded Performance
Partial Outage
Major Outage
Maintenance
Past Incidents
Sep 18, 2018

No incidents reported today.

Sep 17, 2018

No incidents reported.

Sep 16, 2018

No incidents reported.

Sep 15, 2018

No incidents reported.

Sep 14, 2018

No incidents reported.

Sep 13, 2018

No incidents reported.

Sep 12, 2018

No incidents reported.

Sep 11, 2018

No incidents reported.

Sep 10, 2018

No incidents reported.

Sep 9, 2018

No incidents reported.

Sep 8, 2018

No incidents reported.

Sep 7, 2018
Resolved - Everything is green across the board. Thanks for coming along for the ride.

While we don't expect this to be true, shoot an email to support@papertrailapp.com if anything looks out of place.
Sep 7, 13:40 PDT
Monitoring - All the queues are clear and we're 100% live. We'll keep an eye on things to make sure they stay that way. Stay tuned for the conclusion to this epic saga.
Sep 7, 13:27 PDT
Update - There was a slight speed bump in that some folks ended up with larger delays but the news in the end is good. Some will still have a delay of up to 10 minutes while others will have no delay at all. Stay tuned as we're expecting everyone to burn down to zero soon.
Sep 7, 12:00 PDT
Identified - We've discovered an issue regarding logs taking longer than usual to be made available within the Viewer. We're aware of what's causing the delay (an effect of the microcode patches from our recent maintenance window) and working on bolstering the processes to eliminate the delay altogether. Currently, the delay can be between 2 and 5 minutes depending on what slot your account falls into within the pipeline.
Sep 7, 10:58 PDT
Sep 6, 2018
Postmortem - Read details
Sep 12, 17:46 PDT
Resolved - This incident has been resolved.
Sep 6, 20:44 PDT
Monitoring - We are almost done burning the backlog, ETA 1 hour.
Sep 6, 19:38 PDT
Update - There is no ETA at the moment. Log viewer may be 2-3 hours behind.
Sep 6, 13:08 PDT
Update - Disk issues resolved. Moving ETA to 4:00 PM UTC.
Sep 6, 07:56 PDT
Update - We are dealing with disk space issues, moving ETA to 2:30 PM UTC
Sep 6, 05:48 PDT
Update - ETA to resolution is 1:30 PM UTC.
Sep 6, 04:01 PDT
Update - We're 76% done with the backlog, revised ETA is 3 hours from now.
Sep 6, 01:42 PDT
Investigating - We are currently investigating this issue.
Sep 5, 23:57 PDT
Sep 5, 2018
Completed - The scheduled maintenance has been completed.
Sep 5, 23:45 PDT
Update - We are about 70% done with the backlog burndown. ETA to resolution is 3 hours. Will continue providing updates as they come.
Sep 5, 23:24 PDT
Update - Backlog continues to burn down. Will update with more information and ETA to full resolution in 90 minutes.
Sep 5, 21:51 PDT
Update - We've reached the upper bounds of the additional capacity we can add. We're about 60% of the way.
Sep 5, 21:17 PDT
Update - We are provisioning extra capacity on top of the extra capacity to see if we can clear the backlog quicker.
Sep 5, 18:22 PDT
Update - 52% of the backlog is cleared.
Sep 5, 16:56 PDT
Update - Around 47% of the backlog has been cleared.
Sep 5, 15:44 PDT
Update - Datastore backlog is 30% processed, we continue to process the backlog as fast as possible.
Sep 5, 13:10 PDT
Update - Datastore backlog is 23% processed, we continue to process the backlog as fast as possible.
Sep 5, 12:24 PDT
Update - Filtering is now fully operational. We continue to work on the datastore inserter backlog.
Sep 5, 10:38 PDT
Update - 82% of the filtering layer backlog has been processed. Roughly 5% of the log search data backlog has processed.
Sep 5, 09:30 PDT
Update - We are extending the maintenance window as we continue to investigate. Backlog is being processed at the moment.
Sep 5, 07:26 PDT
Update - Catching up with the backlog is taking longer than expected - we are investigating. There will be a delay in log viewing and search alerts.
Sep 5, 05:53 PDT
Update - Capacity is coming back online, we're focusing on burning down any queue lag.
Sep 5, 05:13 PDT
Update - Our provider has taken down more than expected capacity. There is a slight delay in log viewing. We are trying to bring this under control now.
Sep 5, 04:32 PDT
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Sep 5, 03:45 PDT
Scheduled - Our hosting provider will be restarting a significant number of servers during this time window.
Log collection and the website are not impacted. Log Search and Search Alerts performance might be degraded during this time. We are working with our hosting provider to minimize the impact to our service.
Aug 31, 08:18 PDT
Sep 4, 2018

No incidents reported.