Resolved -
On April 9, 2025, between 11:27 UTC and 12:39 UTC, the Pull Requests service was degraded and experienced delays in processing updates. At peak, approximately 1–1.5% of users were affected by delays in synchronizing pull requests. During this period, users may have seen a "Processing updates" message in their pull requests after pushing new commits, and the new commits did not appear in the Pull Request view as expected. The Pull Request synchronization process has automatic retries and most delays were automatically resolved. Any Pull Requests that were not resynchronized during this window were manually synchronized on Friday, April 11 at 14:23 UTC.
This was due to a misconfigured GeoIP lookup file that our routine GitHub operations depended on and led to background job processing to fail.
We mitigated the incident by reverting to a known good version of the GeoIP lookup file on affected hosts.
We are working to enhance our CI testing and automation by validating GeoIP metadata to reduce our time to detection and mitigation of issues like this one in the future.
Apr 10, 00:39 UTC
Update -
Pull Requests is operating normally.
Apr 10, 00:36 UTC
Update -
The team has identified a mitigation and is rolling it out while actively monitoring recovery
Apr 10, 00:19 UTC
Update -
Some users are experiencing delays in pull request updates. After pushing new commits, PRs show a "Processing updates" message, and the new commits do not appear in the pull request view.
Apr 9, 23:39 UTC
Update -
Pull Requests is experiencing degraded performance. We are continuing to investigate.
Apr 9, 23:36 UTC
Investigating -
We are currently investigating this issue.
Apr 9, 23:27 UTC