Resolved -
On March 3, 2026, between 18:46 UTC and 20:09 UTC, GitHub experienced a period of degraded availability impacting GitHub.com, the GitHub API, GitHub Actions, Git operations, GitHub Copilot, and other dependent services. At the peak of the incident, GitHub.com request failures reached approximately 40%. During the same period, approximately 43% of GitHub API requests failed. Git operations over HTTP had an error rate of approximately 6%, while SSH was not impacted. GitHub Copilot requests had an error rate of approximately 21%. GitHub Actions experienced less than 1% impact.
This incident shared the same underlying cause as an incident in early February where we saw a large volume of writes to the user settings caching mechanism. While deploying a change to reduce the burden of these writes, a bug caused every user’s cache to expire, get recalculated, and get rewritten. The increased load caused replication delays that cascaded down to all affected services. We mitigated this issue by immediately rolling back the faulty deployment.
We understand these incidents disrupted the workflows of developers. While we have made substantial, long-term investments in how GitHub is built and operated to improve resilience, we acknowledge we have more work to do. Getting there requires deep architectural work that is already underway, as well as urgent, targeted improvements. We are taking the following immediate steps:
- We have added a killswitch and improved monitoring to the caching mechanism to ensure we are notified before there is user impact and can respond swiftly.
- We are moving the cache mechanism to a dedicated host, ensuring that any future issues will solely affect services that rely on it.
Mar 3, 20:09 UTC
Update -
We're seeing recovery across all services. We're continuing to monitor for full recovery.
Mar 3, 20:06 UTC
Update -
Actions is operating normally.
Mar 3, 19:55 UTC
Update -
Git Operations is operating normally.
Mar 3, 19:54 UTC
Update -
Git Operations is experiencing degraded availability. We are continuing to investigate.
Mar 3, 19:36 UTC
Update -
We are seeing recovery across multiple services. Impact is mostly isolated to git operations at this point, we continue to investigate
Mar 3, 19:33 UTC
Update -
Copilot is operating normally.
Mar 3, 19:31 UTC
Update -
Pull Requests is operating normally.
Mar 3, 19:31 UTC
Update -
Pull Requests is experiencing degraded performance. We are continuing to investigate.
Mar 3, 19:28 UTC
Update -
Issues is operating normally.
Mar 3, 19:27 UTC
Update -
Webhooks is operating normally.
Mar 3, 19:25 UTC
Update -
Codespaces is operating normally.
Mar 3, 19:25 UTC
Update -
Webhooks is experiencing degraded performance. We are continuing to investigate.
Mar 3, 19:24 UTC
Update -
Issues is experiencing degraded performance. We are continuing to investigate.
Mar 3, 19:23 UTC
Update -
We've identified the issue and have applied a mitigation. We're seeing recovery of services. We continue to montitor for full recovery.
Mar 3, 19:17 UTC
Update -
API Requests is operating normally.
Mar 3, 19:15 UTC
Update -
API Requests is experiencing degraded performance. We are continuing to investigate.
Mar 3, 19:14 UTC
Update -
Codespaces is experiencing degraded performance. We are continuing to investigate.
Mar 3, 19:11 UTC
Update -
Pull Requests is experiencing degraded availability. We are continuing to investigate.
Mar 3, 19:05 UTC
Update -
Webhooks is experiencing degraded availability. We are continuing to investigate.
Mar 3, 19:04 UTC
Update -
We're seeing some service degradation across GitHub services. We're currently investigating impact.
Mar 3, 19:03 UTC
Update -
Webhooks is experiencing degraded performance. We are continuing to investigate.
Mar 3, 19:02 UTC
Update -
Pull Requests is experiencing degraded performance. We are continuing to investigate.
Mar 3, 19:00 UTC
Update -
API Requests is experiencing degraded availability. We are continuing to investigate.
Mar 3, 19:00 UTC
Investigating -
We are investigating reports of degraded availability for Actions, Copilot and Issues
Mar 3, 18:59 UTC