Tempo Querier Read Instability
This incident has been resolved.
Incident History
Grafana Cloud Tempo: Intermittent Read Failure
Our engineering team has implemented a fix, and this incident is resolved.
Multi-http checks failing to run in Tokyo probe location
The incident began at UTC 18:35 Tokyo probe was fully operational again as of UTC 19:05 and has been operating normally since then. Resolving the incident.
Degraded Performance in prod-us-central-5
There are been no marked degradation over the last 24 hours. At this time, we are considering this issue resolved. There will be no further updates.
Issues Querying Hosted Logs
This incident has been resolved.
Some Loki Queries Failing
This incident has been resolved. All query function should work as expected.
Newly Created API Keys
We recently encountered an issue where newly-created Grafana Cloud API keys were not authenticating correctly with backend gateway services (e.g., Mimir, Loki, Tempo). This was due to a change implemented late last week. A hotfix has been applied and this issue is fully resolved.
Rule Performance Degradation
There were no observed instances of this issue today. We're considering the incident resolved. Please contact our Support team if you believe you're impacted by this incident.
Synthetics MultiHTTP checks failing in Sao Paulo, Seoul, and Mumbai probe locations
The incident began at 01:30 UTC. We saw a 100% failure rate for multiHTTP checks in Sao Paulo, Seoul, and Mumbai. All other check types are running normally. The incident was resolved at 15:00 UTC
Degraded read/write path for cloud metrics & traces in us-central2
Engineering has released a fix and as of 16:07 UTC, customers should no longer experience read/write path degradation on us-central2 cloud metrics and traces. At this time, we are considering this issue resolved. No further updates.