Errors (5xx, timeouts) in Fly.io dashboard

Resolved

This incident is resolved, Dashboard, API and CLI operations should be working normally now.

Update

We continue to monitor the deployed fix. Dashboard and API/CLI operations should be functional now.

Update

We have identified the troublesome component and a fix has been rolled out. We are monitoring the results and may need to perform further updates to fully stabilize things.

Investigating

Our metrics and user reports show Fly.io/dashboard and portions of the API backend are timing out or returning 5xx errors. All operations in the Fly dashboard and most operations using fly CLI will fail or timeout at this point.

Currently-running machines or workloads should not be affected.