Crashlytics Incident #20002
The Firebase Crashlytics is experiencing delays in crash processing.
Incident began at 2020-04-24 07:30 and ended at 2020-04-29 14:13 (all times are US/Pacific).
| Date | Time | Description | |
|---|---|---|---|
| Apr 29, 2020 | 14:13 | The issue with Firebase Crashlytics has been resolved for all affected projects as of Wednesday, 2020-04-29 14:03 US/Pacific. We thank you for your patience while we've worked on resolving the issue. |
|
| Apr 28, 2020 | 16:45 | Description: Firebase Crashlytics delays in crash processing have been reduced by mitigation done by our engineering team, and non-fatal crashes have resumed processing at a 100% rate. Current Mitigation: Non-fatal crashes remain at a 100% processing rate. We are continuing to monitor the situation to make sure things have fully returned to normal before declaring all-clear. Symptoms: Potential delays in crash reporting. We will provide an update by Wednesday, 2020-04-29 17:00 US/Pacific with current details. We apologize to all who are affected by the disruption. Diagnosis: High latency on crash reporting and missing Android Non-Fatal crash reporting. Workaround: None at the moment. |
|
| Apr 28, 2020 | 13:58 | Description: Firebase Crashlytics delays in crash processing have been reduced by mitigation done by our engineering team. Current Mitigation: Non-Fatal crashes have been returned to a 100% sampling rate. We are monitoring the situation to make sure things have returned to normal before declaring all-clear. Symptoms: Potential delays in crash reporting. We will provide an update by Tuesday, 2020-04-28 17:00 US/Pacific with current details. We apologize to all who are affected by the disruption. Diagnosis: High latency on crash reporting and missing Android Non-Fatal crash reporting. Workaround: None at the moment. |
|
| Apr 27, 2020 | 13:49 | Description: Firebase Crashlytics delays in crash processing have been reduced by mitigation done by our engineering team. Current Mitigation: Non-Fatal crashes are being sampled at a rate of 50%. We plan to increase sampling rates back to 100% as soon as possible. Symptoms: Delays in crash reporting; non-fatal reports are being sampled at a 50% rate. We will provide an update by Tuesday, 2020-04-28 14:00 US/Pacific with current details. We apologize to all who are affected by the disruption. Diagnosis: High latency on crash reporting and missing Android Non-Fatal crash reporting. Workaround: None at the moment. |
|
| Apr 24, 2020 | 14:34 | Description: Firebase Crashlytics delays in crash processing have been reduced by mitigation done by our engineering team. Current Mitigation: All Non-Fatal crashes are being dropped until Monday. Symptoms: Delays in crash reporting; non-fatal reports dropped. We will provide an update by Monday, 2020-04-27 14:00 US/Pacific with current details. We apologize to all who are affected by the disruption. Diagnosis: High latency on crash reporting and missing Android Non-Fatal crash reporting. Workaround: None at the moment. |