Disruption with GitHub services

Resolved

On July 02, 2024, between 18:21 UTC and 19:24 UTC the code search service was degraded and returned elevated 500 HTTP status responses. On average, the error rate was 38% of code search requests. This was due to a bad deployment causing some user's rate limit calculations to error while processing code search requests. This impacted approximately 2,000 users.We mitigated the incident by rolling back the bad deployment along with resetting rate limits for all users.We have identified and implemented updates in the testing of rate limit calculations to prevent this problem from happening again, and clarified deployment processes for verification before a full production rollout to minimize impact in the future.

Investigating

API Requests is operating normally.

Investigating

The fix has been rolled out and our telemetry indicates that the errors with code search have resolved.

Investigating

An issue with faulty data in an in-memory storage causes around one third of code search requests to fail. The team has identified the issue and is working on rolling out a fix.

Investigating

API Requests is experiencing degraded performance. We are continuing to investigate.

Investigating

We are currently investigating this issue.