Incident History

Some Cloud Test Runs Terminated

Between approximately 13:40 and 15:05 UTC today, a subset of cloud test runs were unexpectedly terminated and marked Aborted (by system). This affected runs that were in progress during three short windows in that period; test results and metrics for completed runs were not impacted.

We have identified the cause and no further occurrences have been observed since 15:05 UTC. A fix is in place. The platform is currently operating normally and test runs are executing as expected

1785952635 - 1785952635 Resolved

o11y requests too large for nods in the prod-eu-west-2 region.

This incident has been resolved.

1785930095 - 1785930275 Resolved

Some Grafana Instances Unavailable

We are observing a continued period of stability and at this point, we are marking the incident resolved.

1785870669 - 1785893521 Resolved

Logs latency increase within prod-eu-west-3

This incident has been resolved.

1785839653 - 1785851340 Resolved

K6 - Cloud test-run issues

This incident has been resolved.

1785569138 - 1785573097 Resolved

Partial Read Outage for Loki in prod-us-east-4

We are investigating a partial read outage affecting Loki in prod-us-east-4. Between 19:41 UTC and 19:50 UTC, a significant portion of read queries may have failed or returned errors.

The issue has been identified and service has been restored. We are continuing to investigate the underlying cause and will provide additional information as it becomes available.

1785530020 - 1785530020 Resolved

Degraded Performance: Stack Provisioning Failures within certain reigons (PDC Setup)

He have identified the cause and applied a fix for this issue and all effected stacks within the affected regions are not working as expected.

1785495089 - 1785500011 Resolved

PDC Authentication Issues

This incident has been resolved.

1785422731 - 1785433215 Resolved

Issues with Billing/Usage Dashboard Metrics and Panels.

This incident has been resolved.

1785417350 - 1785424658 Resolved

IRM Performance Degradation in EU Region

The issue affecting Grafana IRM in the EU region has been resolved. The IRM UI, public API, and alert notification processing have been restored and are operating normally.

1785361603 - 1785371966 Resolved
⮜ Previous Next ⮞