Incident with Pull Request Rebase Merges

Resolved

On January 16, 2025, between 00:45 UTC and 09:40 UTC the Pull Requests service was degraded and failed to generate rebase merge commits. This was due to a configuration change that introduced disagreements between replicas. These disagreements caused a secondary job to run, triggering timeouts while computing rebase merge commits. We mitigated the incident by rolling back the configuration change.We are working on improving our monitoring and deployment practices to reduce our time to detection and mitigation of issues like this one in the future.

Investigating

The incident has been resolved, but please note affected pull requests will self repair when any commits are pushed to the pull requests' base branch or head branch. If you encounter problems with a rebase and merge, either click the "update branch" button or push a commit to the PR's branch.

Investigating

We have mitigated the incident, and any new pull request rebase merges should be recovered. We are working on recovery steps for any pull requests that attempted to merge during this incident.

Investigating

We believe to have found a root cause, and in the process of verifying the mitigation.

Investigating

We are still continuing to investigate.

Investigating

We are still experiencing failures for rebase merges in pull requests, we are continuing to investigate.

Investigating

We are investigating reports of degraded performance for Pull Requests