Incident Period: March 10th, ~8:00 PM (GMT)
Issue: Some of our services became unavailable in the UAE, causing a significant number of incoming requests to fail.
Root Cause: Services were unable to connect to their database after the database server certificates were rotated.
Resolution: The certificate bundle on the services was updated.
Services were unable to connect to the database after a certificate rotation, as the new certificate was issued by a certificate authority that was not trusted by our service.
We updated our service's certificate bundle to include the new certificate authority, and are updating our internal processes to ensure we are better aware of future certificate rotations.