Incident with Webhooks


Incident resolved in 31m50s

Resolved

On August 12, 2024 from 13:39 to 14:28 UTC some users experienced an elevated rate of errors of up to 0.45% from the GitHub API. Less than 5% of webhooks interactions failed and less than 0.5% of Actions runs were delayed.This impact was caused by internal networking instances being insufficiently scaled.We mitigated the incident by provisioning additional instances. We are working to enhance the sizing strategy for the relevant infrastructure to prevent similar issues and to also improve monitoring and processes to reduce our time to detection and mitigation of issues like this one in the future.

1723473666

Investigating

Webhooks is experiencing degraded performance. We are continuing to investigate.

1723473665

Investigating

Webhooks is operating normally.

1723473664

Investigating

Some customers may see failures calling the webhooks API or running queries for enterprise or organization audit logs. We have started mitigating the issue and are watching recovery. We will provide an update within the next 30 minutes.

1723473125

Investigating

We are investigating reports of degraded performance for Webhooks

1723471756