Incident with Actions


Incident resolved in 1h1m36s

Resolved

On August 21, 2024, between 13:48 UTC and 15:00 UTC, Actions experienced degraded performance, leading to delays in workflow runs. On average, 25% of workflow runs were delayed by 8 minutes. Less than 1% of workflow runs exhausted retries and failed to start. The issue stemmed from a backlog of Pull Request events which caused delays in Actions processing the event queues that trigger workflow runs.We mitigated the incident by disabling the process that led to the sudden spike in Pull Request events. We are working to improve our monitoring and deployment practices to reduce our time to detection and mitigation of issues like this one in the future. We are also identifying appropriate changes to rate limits and reserved capacity to reduce the breadth of impact.

1724253066

Investigating

We have seen recovery and Actions workflow runs are now running as expected.

1724253057

Investigating

We are seeing reduced delays for Actions workflow runs to get triggered. We are continuing to investigate how to further reduce impact on customers and recover more quickly.

1724251896

Investigating

We are investigating reports of users seeing delays for Actions workflow runs to get triggered.

1724249753

Investigating

We are investigating reports of degraded performance for Actions

1724249370