We recently encountered an issue where newly-created Grafana Cloud API keys were not authenticating correctly with backend gateway services (e.g., Mimir, Loki, Tempo). This was due to a change implemented late last week. A hotfix has been applied and this issue is fully resolved.
There were no observed instances of this issue today. We're considering the incident resolved. Please contact our Support team if you believe you're impacted by this incident.
The incident began at 01:30 UTC. We saw a 100% failure rate for multiHTTP checks in Sao Paulo, Seoul, and Mumbai. All other check types are running normally. The incident was resolved at 15:00 UTC
Engineering has released a fix and as of 16:07 UTC, customers should no longer experience read/write path degradation on us-central2 cloud metrics and traces. At this time, we are considering this issue resolved. No further updates.
This incident has been resolved.
Engineering has released a fix and as of 10:30 UTC, customers should no longer experience hosted services issues. At this time, we are considering this issue resolved. No further updates.
All clusters have been restarted and datasources should be fully operational.
Starting at ~21:11 UTC, Loki had a brief outage. Users may have experienced degraded write path performance for approximately 10 minutes, and 2-3 minutes of total write and read outage. This has since been resolved.
This incident has been resolved.
This incident has been resolved.