Incident History

MongoDB Cloud: Slow page loads

This incident has been resolved.

Jun 11, 16:52 - Jun 11, 17:08 Resolved

MongoDB Atlas Online Archive: Users may not be able to access Online Archive data

This incident has been resolved.

Jun 04, 16:42 - Jun 04, 18:33 Resolved

Delayed cluster modifications

Incident Summary

Between June 3rd 2024, 18:24 UTC and June 4th, 1:21 UTC, MongoDB Atlas customers across all commercial regions experienced degradation of the following functionalities.

Changes to cluster configuration, both in response to API calls and automated triggers (e.g., auto-scaling or node replacement) were executed with delays varying between a few minutes and several hours. Similar delays applied to Alert processing, impacting both the Atlas built-in Alert functionality and integrations such as Datadog and PagerDuty. Part of the daily cost tabulation for compute, storage and data transfer on June 3rd will be delayed to June 6th or June 7th. Lower granularities of cluster metrics were unavailable until up to 1:56 UTC. Finally, the Data Explorer and Realtime Performance Panel UIs were impaired for several hours.

Root Cause

The root cause of this incident was an unexpected failure during an Atlas internal maintenance activity. A planned migration of metadata for an internal workflow management system that backs the Atlas control plane resulted in unexpected resource congestion on the target database cluster as traffic was redirected to the target. This issue did not occur during pre-production testing. The rollback process required complex reconciliation of data in the source and target databases, delaying recovery.

MongoDB Actions

The MongoDB Atlas team is designing a new metadata migration strategy for future maintenance activities. This process will allow for rollback within less than 15 minutes of detecting a potential issue. We will not execute similar maintenance activities until this procedure is implemented and thoroughly tested.

Recommended Customer Actions

This issue was fully eliminated by the MongoDB Atlas team and does not require customer action.

Jun 03, 18:45 - Jun 04, 02:11 Resolved

MongoDB Cloud: Occasional page load errors

This incident has been resolved.

May 23, 21:26 - May 23, 21:39 Resolved

MongoDB Atlas for Government: Cluster Operations Delayed

This issue has been fully resolved. Thank you for your patience.

May 13, 15:21 - May 13, 15:47 Resolved

Atlas Data Federation and Online Archive users may see increased query timeouts in AWS/ap-southeast-1

This incident has been resolved.

May 09, 18:05 - May 09, 19:16 Resolved

MongoDB Atlas: Invoice summaries are not loading

This incident has been resolved.

May 06, 17:19 - May 06, 17:54 Resolved

MongoDB Atlas and Cloud Manager: Degraded Performance for Atlas and Cloud Manager UI and API Operations

This incident has been resolved.

May 06, 16:38 - May 06, 17:01 Resolved

Data Federation errors when querying user-owned Azure containers

This incident has been resolved.

May 02, 19:58 - May 02, 20:25 Resolved

Degraded Performance: Atlas UI and API Operations

We have resolved the issue and all systems are operating normally

May 02, 00:33 - May 02, 01:26 Resolved
⮜ Previous Next ⮞