Delays in log processing
Incident Report for Papertrail
Resolved
This incident has been resolved.
Posted Oct 01, 2018 - 13:09 PDT
Update
Backlogs for 80% of customers have cleared. Continuing to focus on the remaining few.
Posted Oct 01, 2018 - 12:59 PDT
Identified
The root cause for this degradation was identified and cleaned up.
Posted Oct 01, 2018 - 12:40 PDT
Update
50% of customers are now at real-time status. The backlogs for the remaining are continuing to progress nicely.
Posted Oct 01, 2018 - 12:40 PDT
Update
Verified all logs are being ingested in real-time and without issue. Marking Log Collection as Operational.

About 20% of customers should be back to real-time. The remainder are heading in the right direction toward real-time.
Posted Oct 01, 2018 - 12:09 PDT
Investigating
The queue has grown a bit to 15 minutes in total size but logs are actively being processed. SRE is continuing to monitor.
Posted Oct 01, 2018 - 11:39 PDT
Identified
We have identified the cause of the bottleneck and are now burning down the backlog.
Posted Oct 01, 2018 - 11:29 PDT
Update
Some customers are seeing a delay of 10 minutes for log ingestion and search. Our SRE team is engaged and actively clearing the backlog.
Posted Oct 01, 2018 - 11:16 PDT
Investigating
We are currently investigating this issue.
Posted Oct 01, 2018 - 11:12 PDT
This incident affected: Log Collection and Log Search.
← Current Status Powered by Atlassian Statuspage