Some tweaks were made and bolts were tightened. All customers are back to their original positions. We're keeping a watchful eye on this but don't expect this to be an issue going forward.
We're putting this under observation, rolling everyone back into normal space in waves, and continuing to monitor for any new leaks.
Posted Oct 02, 2018 - 16:12 PDT
Update
Engineering is still investigating this. No new news to report.
Posted Oct 02, 2018 - 13:44 PDT
Update
We don't have any new information to report at this time. We're still investigating the root cause. Papertrail continues to function normally.
Posted Oct 02, 2018 - 11:55 PDT
Investigating
Less than 1/10th of 1% of users are reporting seeing errant logs in the Events viewer. As a precaution, while we monitor this and track down a possible cause, we've migrated everyone to a different processing pipeline. The side effects include temporarily losing the velocity graphs at the bottom of the page and longer search times. All logs are being ingested and processed properly.
This is absolutely temporary and just a safety precaution at this time. Once we're confident this is no longer an issue, we'll move customers back to where they were. The rest of Papertrail should function entirely the same (alerts, the API, etc.).