Firebase Status Dashboard
NOTE
For incidents related to Cloud Functions, Cloud Firestore and Cloud Storage, please see Cloud Status Dashboard. And for incidents related to Google Analytics, please see Ads Status Dashboard.
Incident affecting Hosting
We've received a report of an issue with Firebase Hosting.
Incident began at 2020-04-21 11:43 and ended at 2020-04-21 13:31 (all times are US/Pacific).
Date | Time | Description | |
---|---|---|---|
| 19 May 2020 | 10:32 PDT | Incident Analysis Impact: Most noncached requests made against the Hosting origin were returning a 503 instead of serving content between 2020-04-21 11:43 and 2020-04-21 13:31 (all times are US/Pacific). Root Cause: A large amount of a specific type of request traffic was not cached by our CDN for a single customer. All of these cache misses resulted in a huge spike in requests served by our origin service. Once this service was fully overloaded it began serving 503s to all customers. Resolution: The offending domain was blocked and appropriate preventative action was put in place to avoid similar issues in the future, examples including: additional log reporting, additional monitoring, and bug fixes in caching layer. |
| 21 Apr 2020 | 13:31 PDT | The issue with Firebase Hosting has been resolved for all affected users as of Tuesday, 2020-04-21 12:39 US/Pacific. We will publish an analysis of this incident once we have completed our internal investigation. We thank you for your patience while we've worked on resolving the issue. |
| 21 Apr 2020 | 12:37 PDT | Description: Mitigation work is currently underway by our engineering team. We have made a change and error rates are coming down quickly. We will provide more information by Tuesday, 2020-04-21 13:33 US/Pacific. Diagnosis: High 503 error rate for Firebase Hosting traffic. Workaround: None at this time. |
| 21 Apr 2020 | 11:58 PDT | Description: We are investigating elevated 503 rates for Firebase Hosting traffic. Our engineering team continues to investigate the issue. We will provide an update by Tuesday, 2020-04-21 13:00 US/Pacific with current details. We apologize to all who are affected by the disruption. Diagnosis: High 503 error rate for Firebase Hosting traffic. Workaround: None at this time. |
| 21 Apr 2020 | 11:53 PDT | Description: We are investigating elevated 503 rates for Firebase Hosting traffic. Our engineering team continues to investigate the issue. We will provide an update by Tuesday, 2020-04-21 12:54 US/Pacific with current details. We apologize to all who are affected by the disruption. |
| 21 Apr 2020 | 11:47 PDT | Description: We've received a report of an issue with Firebase Hosting as of Tuesday, 2020-04-21 11:43 US/Pacific. We will provide more information by Tuesday, 2020-04-21 12:15 US/Pacific. |
- All times are US/Pacific