Incident History

Major GCP Incident Affecting Multiple Grafana Cloud Components (Including AWS and Azure deployed Instances)

We continue to observe a continued period of recovery. At this time, we are considering this issue resolved. No further updates.

Loki - Slow Queries

We continue to observe a continued period of recovery. At this time, we are considering this issue resolved. No further updates.

Instances on the "Slow" Release Channel Receiving Unexpected Errors.

At approximately 12:00 UTC a feature toggle was rolled out which negatively impacted instances on the slow release channel. Users on this release channel began to receive an "AlertStatesDataLayer" error. A workaround was quickly identified and applied to reporting users. The feature toggle in question was fully reverted by 18:00 UTC.

Private Datasource Connect - New Agents Failing To Get SSH Certificates Signed

We continue to observe a continued period of recovery. At this time, we are considering this issue resolved. No further updates.

Scheduled maintenance caused a temporary issue with logging in into Grafana Cloud stacks.

Due to scheduled maintenance (https://status.grafana.com/incidents/rz7nt6cs4prb) we hit an issue with some users being unable to log in into their Grafana Cloud stacks. The issue was affecting only users who:

had no session already opened in the Grafana Cloud stack;
or they were located close to Europe (geographically), but their stack is closer to US (or vice versa). The issue was caused by an incorrect configuration introduced by the maintenance, which was fixed shortly after being discovered. Login is fully operational and stable now.