Test Lab Incident #19015
We are experiencing an issue with Firebase Test Lab
Incident began at 2019-12-11 11:55 and ended at 2019-12-11 13:07 (all times are US/Pacific).
| Date | Time | Description | |
|---|---|---|---|
| Dec 11, 2019 | 13:07 | The issue with elevated pending time for Firebase Test Lab virtual devices has been resolved for all affected users as of Wednesday, 2019-12-11 12:59 US/Pacific. We thank you for your patience while we've worked on resolving the issue. |
|
| Dec 11, 2019 | 12:42 | Description: A rollback of a key system is complete. It appears that the number of tests pending is now reducing, but is still elevated. Our engineering team is continuing to monitor to evaluate the success of this mitigation. Symptoms: Customers may experience longer than normal pending states for tests on android virtual devices. We will provide more information by Wednesday, 2019-12-11 13:50 US/Pacific. Diagnosis: Customers may experience longer than normal pending states for tests on android virtual devices. Workaround: None at this time. |
|
| Dec 11, 2019 | 12:27 | Description: To mitigate, our engineering team is rolling back a change suspected to cause this issue. We do not have an ETA for mitigation at this point. Symptoms: Customers may experience tests waiting in pending state for longer than normal. We will provide more information by Wednesday, 2019-12-11 13:30 US/Pacific. Diagnosis: Customers may experience tests waiting in pending state for longer than normal. Workaround: None at this time. |
|
| Dec 11, 2019 | 12:12 | Description: To mitigate, our engineering team is rolling back a change suspected to cause this issue. We do not have an ETA for mitigation at this point. We will provide more information by Wednesday, 2019-12-11 13:15 US/Pacific. Diagnosis: Customers may experience inconclusive results for virtual tests. Workaround: None at this time. |
|
| Dec 11, 2019 | 11:58 | Description: We are experiencing an issue with Firebase Test Lab, beginning at Wednesday, 2019-12-11 10:40 US/Pacific. Symptoms: Customers may experience inconclusive results for virtual tests. Our engineering team is investigating the issue. We will provide an update by Wednesday, 2019-12-11 13:00 US/Pacific with current details. We apologize to all who are affected by the disruption. |