Authentication issues related to API requests
Resolved
Between 15:05 UTC and 16:25 UTC, GitHub API services experienced degraded availability due to sporadic authentication failures affecting approximately 9% of requests. Customers experienced intermittent "logged out" behavior as erroneous 401 responses triggered repeated authentication flows in app integrations. Affected requests also experienced approximately 800ms of additional latency. A memcached proxy service rollout to our internal API infrastructure caused our authentication service to pick up an incorrect memcached host configuration, leading to intermittent authentication lookup failures. We mitigated the incident by deploying a configuration change to memcached to use the correct host. To prevent similar issues in the future, we plan to migrate our authentication system to the new memcached infrastructure to improve resilience and strengthen overall reliability posture.
Update
The degradation has been mitigated. We are monitoring to ensure stability.
Investigating
The degradation affecting API Requests has been mitigated. We are monitoring to ensure stability.
Investigating
We continue to investigate issues related to sporadic authentication failures, impacting approximately 15% of API traffic. Erroneous 401 responses are causing app integrations to trigger authentication flows. We have identified a problematic component in our infrastructure and are working to mitigate.
Investigating
The degradation affecting Issues has been mitigated. We are monitoring to ensure stability.
Investigating
We continue to investigate issues related to sporadic authentication failures, impacting approximately 15% of API traffic. Further updates will be provided as we work to mitigate.
Investigating
We are investigating issues related to sporadic authentication failures impacting approximately 15% of API traffic. We will continue to investigate and provide updates.
Investigating
Issues is experiencing degraded performance. We are continuing to investigate.
Investigating
API Requests is experiencing degraded availability. We are continuing to investigate.
Investigating
We are investigating reports of impacted performance for some GitHub services.