Incident with search on GitHub we are seeing increased failure rates

Resolved

On August 12, 2025, between 13:30 UTC and 17:14 UTC, GitHub search was in a degraded state. Users experienced inaccurate or incomplete results, failures to load certain pages (like Issues, Pull Requests, Projects, and Deployments), and broken components like Actions workflow and label filters.Most user impact occurred between 14:00 UTC and 15:30 UTC, when up to 75% of search queries failed, and updates to search results were delayed by up to 100 minutes. The incident was triggered by intermittent connectivity issues between our load balancers and search hosts. While retry logic initially masked these problems, retry queues eventually overwhelmed the load balancers, causing failure. The query failures were mitigated at 15:30 UTC after throttling our search indexing pipeline to reduce load and stabilize retries. The connectivity failures were resolved at 17:14 UTC after the automated reboot of a search host, causing the rest of the system to recover. We have improved internal monitors and playbooks, and tuned our search cluster load balancer to further mitigate the recurrence of this failure mode. We continue to invest in resolving the underlying connectivity issues.

Investigating

Service availability has been mostly restored, but some users will continue to see increased request latency and stale search results. We are still working towards full recovery.

Investigating

Service availability has been mostly restored, but increased load/query latency and stale search results persist. We continue to work towards full mitigation.

Investigating

We are seeing partial recovery in service availability, but still see inconsistent experiences and stale search data across services. Investigation and mitigations are underway.

Investigating

We are experiencing increased latency in our API layers and inconsistently degraded experiences when loading or querying issues, pull requests, labels, packages, releases, workflow runs, projects, and repositories, among others. Investigation is underway.

Investigating

We are investigating reports of degraded performance in services backed by search. The team continues to investigate why requests are failing to reach our search clusters.

Investigating

Packages is experiencing degraded performance. We are continuing to investigate.

Investigating

We are investigating reports of degraded performance for API Requests, Actions, Issues and Pull Requests