Creation of Load Balancers and Mongo Managed Databases
Resolved
Our Engineering team has completed remediation of the previously stalled Mongo clusters and those clusters are now online.
This incident is now fully resolved. If you have any questions or continue to experience issues, please reach out to Support from within your account.
Thank you for your patience throughout this incident.
Update
From 16:24 - 18:44 UTC, Load Balancers and Mongo Managed Databases failed to create successfully. Users experienced success messages, but the create processes did not complete as expected.
As of 18:44 UTC, users are able to create Load Balancers and Mongo clusters normally.
As of 19:00 UTC, all stalled Load Balancer creation operations were completed.
Our Engineering team is working to remediate a small number of stalled Mongo cluster creation attempts from the incident impact period. Users may await these remediation efforts, or simply create a new cluster, which will be successful.
We will post a final update once all stalled clusters have been remediated.
Update
The potential fix has completed deployment and has been confirmed to address the root cause of this incident. Users should now be able to create Load Balancers and Mongo Managed Database. Our Engineering team is monitoring the situation and we will post an update as soon as we confirm the issue is fully resolved.
Update
After further investigation, our Engineering team has confirmed that only Mongo Managed Database creates are impacted, along with Load Balancers. Managed Kubernetes is not impacted by this incident. A potential fix is currently in progress. We will provide another update as soon as possible.
Investigating
Our Engineering team is investigating an issue with creation of Load Balancers, Managed Databases, and Managed Kubernetes clusters in all regions, via both the Cloud Control Panel and API. During this time, users may see success messages when initially creating resources, but those requests will not complete. We apologize for the inconvenience and will share an update once we have more information.