Grafana log search not loading

Resolved

This incident has been resolved.

Update

New app logs are once again stably flowing into the fly-metrics.net dashboard. Users should see and be able to search all app logs from 22:30 UTC onwards. We are continuing to monitor the customer logs cluster to ensure continued stability.

Not all historical logs were able to be backfilled. Many users will see missing or incomplete logs for the period between 08:00 - 22:30 UTC Oct-06-2025.

Update

We are continuing to backfill historic logs into fly-metrics.net. New logs will be accessible once the backfill is complete.

Update

We have applied a fix and historic logs are now backfilling into fly-metrics.net. New logs will be accessible once the backfill is complete.

Customers can continue view live logs for their apps using the flyctl logs command and from the logs tab of their app page at Fly.io/dashboard.

Update

We are continuing to work on a fix for this issue. Searchable logs in the fly-metrics.net Grafana board remain unavailable at this time.

Customers can view live logs for their apps using the flyctl logs command and from the logs tab of their app page at Fly.io/dashboard .

Update

The issue has been identified and a fix is being implemented.

Investigating

Grafana is returning 5xx errors when loading logs.