Increased network latency in North America

Resolved

This incident has been resolved.

Update

Our upstream paths have been fixed. We are monitoring the results.

Update

We are working with our upstream network provider to address periodic loss of connectivity over transit in ord

Update

We are seeing a recurrence of elevated latency across some hosts in ORD impacting the MPG control plane and a subset of clusters there. We are working to address this.

Update

The network between our regions has been performing well for the majority of traffic. We're still continuing to monitor a few impacted routes in North America that may be seeing elevated latency and packet loss.

Update

Impacted NA backbones have been sidestepped where possible, and we're continuing to monitor network health.

Update

Managed Postgres in ORD has returned to normal operation.

We continue to see slightly elevated latencies and loss over transits in North America. We are working with our upstream network providers to improve performance.

Investigating

We are investigating elevated network instability across some hosts in ORD. Apps and managed postgres clusters on impacted hosts may see elevated latency or networking errors at this time.