Complete outage in prod-me-central-1

Investigating

We are actively monitoring the situation, but at this time there are no new updates to share. The next update will be provided once we have more information to share. Please reach out to our Support team if you have any questions.

Investigating

We are continuing to investigate this issue.

Investigating

Please continue to refer to the AWS status page for more detailed updates specific to AWS. https://health.aws.amazon.com/health/status

AWS are recommending that affected customers move workloads to alternate regions, and we are recommending the same.

Customers who are impacted and who cannot wait for a restoration of service are asked to:

1. Create a Grafana Cloud stack in an alternate region
2. Update clients to send telemetry to the new region, if using Grafana Alloy then you can use Fleet Management https://grafana.com/docs/grafana-cloud/send-data/fleet-management/introduction/
3. If your instance remains available and you have not configured your dashboards as code, then you may be able to use grafanactl to migrate dashboards https://grafana.com/docs/grafana/latest/as-code/observability-as-code/grafana-cli/grafanacli-workflows/ https://grafana.github.io/grafanactl/

We are continuing to work with our CSP at this time, and will provide updates as they are available.

Investigating

AWS are recommending that affected customers move workloads to alternate regions https://health.aws.amazon.com/health/status and we are recommending the same.

Customers who are impacted and who cannot wait for a restoration of service are asked to:

1. Create a Grafana Cloud stack in an alternate region
2. Update clients to send telemetry to the new region, if using Grafana Alloy then you can use Fleet Management https://grafana.com/docs/grafana-cloud/send-data/fleet-management/introduction/
3. If your instance remains available and you have not configured your dashboards as code, then you may be able to use grafanactl to migrate dashboards https://grafana.com/docs/grafana/latest/as-code/observability-as-code/grafana-cli/grafanacli-workflows/ https://grafana.github.io/grafanactl/

We will provide updates when we have them, but we do not have an expected resolution time at this point.

Investigating

Customers are recommended to configure a new blank stack in an alternative Grafana Cloud region and to reconfigure their clients (such as Grafana Alloy) to send telemetry to that region, Fleet Management can be used for this purpose https://grafana.com/docs/grafana-cloud/send-data/fleet-management/introduction/

Investigating

We are updating this incident to reflect a complete outage in prod-me-central-1, due to an on-going AWS UAE data center issue. We will provide further updates accordingly.

Investigating

We are observing write and read outage errors across all databases (metrics, logs, traces) in prod-me-central-1, due to an on-going AWS UAE data center issue. We will provide further updates accordingly.

Investigating

We are observing write and read outage errors across all databases (metrics, logs, traces) in prod-me-central-1, due to an on-going AWS UAE data center issue. We will provide further updates accordingly.

Investigating

We are seeing elevated write and read path errors in prod-me-central-1, due to an on-going AWS UAE data center issue. We will provide further updates accordingly.