Realtime Database Incident #18001
Investigating an issue with Realtime Database.
Incident began at 2018-01-09 14:30 and ended at 2018-01-09 22:40 (all times are US/Pacific).
| Date | Time | Description | |
|---|---|---|---|
| Jan 09, 2018 | 23:30 | The issue has been solved at 10:40 PM US/Pacific. We found an infrastructure network misconfiguration that is suspected to be the culprit. We'll perform investigation in the coming days to confirm that and figure out ways to mitigate in the future. |
|
| Jan 09, 2018 | 19:24 | The system has again recovered at around 6:50 PM US/Pacific. However, the increased latency is being encountered intermittently and the team is still monitoring future recurrences. |
|
| Jan 09, 2018 | 18:41 | The issue has been encountered again around 6:20 PM US/Pacific. We will provide another status update as soon as possible. |
|
| Jan 09, 2018 | 18:16 | The system has recovered at around 5:30 PM US/Pacific, but we're still looking into the culprit and it may recur at any time. We'll continue to monitor and provide any updates. |
|
| Jan 09, 2018 | 17:48 | We're observing increased latency that started around 2:30 PM US/Pacific but intensified around 4:25 PM US/Pacific that affects all customers. Some instances with high load may observe timeouts and unavailability. We will provide more information as soon as possible. |