Resolved -
Between July 28, 2025 16:31 UTC to July 29, 2025 12:05 UTC users saw degraded Git Operations for raw file downloads. On average, the error rate was .005%, with a peak error rate of 3.89%. This was due to a sustained increase in unauthenticated repository traffic.
We mitigated the incident by applying regional rate limiting, rolling back a service that was unable to scale with the additional traffic, and addressed a bug that impacted the caching of raw requests. Additionally, we horizontally scaled several dependencies of the service to appropriately handle the increase in traffic.
We are working on improving our time to detection and have implemented controls to prevent similar incidents in future.
Jul 29, 01:23 UTC
Update -
We continue to work to mitigate this issue.
Jul 29, 01:02 UTC
Update -
We are investigating additional ways to mitigate this issue.
Jul 29, 00:12 UTC
Update -
We continue to work to mitigate this issue.
Jul 28, 23:16 UTC
Update -
Some customers continue to experience errors when accessing raw files and archives. We are working on a mitigation to address the issue.
Jul 28, 21:50 UTC
Update -
We are actively setting up additional rate limiting to address increased requests from scraping and investigating the need to add additional hosts.
Jul 28, 21:06 UTC
Update -
Provisioning of new hosts is underway. We are still investigating other fixes.
Jul 28, 19:29 UTC
Update -
We are adding additional capacity to our infrastructure to mitigate this issue while still investigating.
Jul 28, 18:55 UTC
Update -
We are still actively investigating this issue.
Jul 28, 18:11 UTC
Update -
Git Operations is experiencing degraded performance. We are continuing to investigate.
Jul 28, 17:30 UTC
Update -
We are investigating errors affecting some archive and raw file downloads. Users may experience rate limit warnings or server errors until this is resolved.
Jul 28, 17:17 UTC
Investigating -
We are currently investigating this issue.
Jul 28, 16:50 UTC