Disruption with some GitHub services


Incident resolved in 46m19s

Resolved

On August 27, 2025 between 20:35 and 21:17 UTC, Copilot, Web and REST API traffic experienced degraded performance. Copilot saw an average of 36% of requests fail with a peak failure rate of 77%. Approximately 2% of all non-Copilot Web and REST API traffic requests failed.

This incident occurred after we initiated a production database migration to drop a column from a table backing copilot functionality. While the column was no longer in direct use, our ORM continued to reference the dropped column. This led to a large number of 5xx responses and was similar to the incident on August 5th. At 21:15 UTC, we applied a fix to the production schema and by 21:17 UTC, all services had fully recovered.

While repairs were in progress to avoid this situation, they were not completed quickly enough to prevent a second incident. We have now implemented a temporary block for all drop column operations as an immediate solution while we add more safeguards to prevent similar issues from occurring in the future. We are also implementing graceful degradation so that Copilot issues will not impact other features of our product.

1756330078

Investigating

API Requests and Issues are operating normally.

1756330067

Investigating

We've discovered the cause of the service disruption and applied a mitigation.

1756329958

Investigating

We are continuing to investigate this issue.

1756329206

Investigating

API Requests is experiencing degraded performance. We are continuing to investigate.

1756328287

Investigating

The team is aware of the root cause of this issue and is working to mitigate the issue quickly.

1756328134

Investigating

Issues is experiencing degraded performance. We are continuing to investigate.

1756327845

Investigating

API Requests is experiencing degraded availability. We are continuing to investigate.

1756327738

Investigating

We are currently investigating this issue.

1756327299