AWS Cluster DNS Changes Degraded Performance

Resolved

DNS updates for ACME are confirmed healthy. Cluster updates are now proceeding as expected.

Update

We are continuing to monitor for any further issues.

Update

We've identified a bad piece of metadata causing DNS issues. We've since patched this metadata and are seeing recovery in ACME updates for cluster topology changes. We will continue to monitor recovery.

Investigating

We are still seeing cases where new cluster update requests are failing to go through due to ACME related DNS issues. We are actively investigating this issue.

Update

Mitigation is in progress; recovery for DNS operations and downstream cluster topology changes has begun.

Update

DNS changes for AWS clusters are currently degraded. Cluster topology modifications may be delayed or not able to complete successfully. We've identified the issue and are pushing out a patch.

Update

DNS changes for AWS clusters are currently degraded. We've identified the issue and are pushing out a patch.