Network issues in AMS region

Resolved

This incident has been resolved.

Update

We are continuing to monitor for any further issues.

Update

Networking on the impacted hosts has been restored. Machines and apps on those hosts will now be reachable. We're continuing to monitor to ensure everything remains stable.

Update

The hardware switchover is complete. We are continuing the process of re-connecting the downed hosts to the network.

Update

Installation of the new hardware has completed and we are starting the switchover process. A networking blip may be observed on Machines in the AMS region during this process.

Update

Installation of the replacement hardware is still ongoing.

Update

Replacement hardware is onsite and is being installed.

Update

The upstream provider has identified this issue to a broken switch, and are working to replace the switch. They expect connectivity to return in ~1 hour.

Investigating

Various hosts in AMS region have lost network connectivity. We are investigating this along with our upstream provider.