All delayed alerts have been sent. We'll continue to monitor the system's performance before declaring it resolved.
Posted Aug 30, 2018 - 09:24 PDT
Update
Log search is caught up. We continue to process alerts.
Posted Aug 30, 2018 - 09:12 PDT
Update
Delays are now affecting around 5% of customers, up to 20 minutes. Delayed alerts are still being sent.
Posted Aug 30, 2018 - 09:08 PDT
Update
We continue to process the backlog. Around 30% of customers are still delayed by up to 90 minutes.
Posted Aug 30, 2018 - 07:26 PDT
Update
We are trying to accelerate backlog completion traffic ramps up. Affected hosts keep exhibiting great performance, despite the increased load.
Posted Aug 30, 2018 - 06:05 PDT
Update
We are continuing to monitor for any further issues.
Posted Aug 30, 2018 - 03:41 PDT
Monitoring
Connections are back to normal and we are burning down the backlog. The fix seems to brought a pronounced benefit to our database.
Posted Aug 30, 2018 - 03:40 PDT
Identified
We have identified a potential fix for the database problems.
Posted Aug 30, 2018 - 03:04 PDT
Update
Inbound connections have stabilized. We are waiting for our provider to give us updates. We are burning down the backlog of messages we've received.
Posted Aug 30, 2018 - 02:15 PDT
Update
We have engaged our networking provider to assist us in troubleshooting.
Posted Aug 30, 2018 - 00:54 PDT
Update
Connections are starting to normalize. We continue investigating and dealing with the backlog. Expect new syslog endpoints creation to be delayed by an unknown time.
Posted Aug 30, 2018 - 00:51 PDT
Update
We are still investigating. We are seeing more successful connections coming in.
Posted Aug 29, 2018 - 23:55 PDT
Update
Log clients are continuing to have connection issues
Posted Aug 29, 2018 - 21:17 PDT
Investigating
We are currently investigating this issue.
Posted Aug 29, 2018 - 20:57 PDT
This incident affected: Log Collection and Log Search.