Tempo - Metrics Generator Outage


Incident resolved in 33m38s

Resolved

The issue affecting Tempo (prod-us-east-0 & prod-us-west-0) and Loki (prod-us-central-5) has been fully resolved.

Metrics generation is now operating normally across all regions, and we continue to monitor for stability.

1761923092

Update

We’ve identified that Loki clusters in prod-us-central-5 were also affected by the same underlying issue impacting Tempo in prod-us-east-0.

Querying and ingestion remained fully operational throughout. The issue has been mitigated and services are stabilizing.

1761923052

Update

We’re investigating an issue in Tempo prod-us-east-0 and prod-us-west-0 that began around 19:00 UTC on Oct 30, causing a total outage for metrics generation. 



This affects Tempo metrics generation, but Tempo query and ingestion remain fully operational.

Engineering is actively engaged and assessing the issue. We will provide updates accordingly.

1761921074