Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prepare for Docker Hub limits #1592

Open
melange396 opened this issue Jan 30, 2025 · 0 comments
Open

Prepare for Docker Hub limits #1592

melange396 opened this issue Jan 30, 2025 · 0 comments
Labels
chore devops building, running, deploying, environment stuff, handy utils, repository-related, engineer QoL, etc tests integration & unit testing, bug triage and prevention

Comments

@melange396
Copy link
Collaborator

In doing a recent release of delphi-epidata, during the CI "build" process, an error was encountered from docker stating "You have reached your pull rate limit. You may increase the limit by authenticating and upgrading" (more complete error text can be found in a collapsed section at the bottom of this message). Docker's documentation suggests this should not have affected us ("The Docker Hub plan limits will take effect on March 1, 2025"), so I am guessing that they prematurely, accidentally, and temporarily enabled rate limiting. A subsequent re-run of the build ran successfully.

If [when] a similar error occurs in the future (the Docker limits are scheduled to be enforced starting 4 weeks from today), in addition to creating/upgrading our Docker Hub user tier(*), we may have change our GitHub Actions workflows to do a Docker login step (like we do for the GitHub Container Registry) before any operations like the one that errored.

Alternatively, we might potentially be able to switch to self-hosted runners and deal with the unauthenticated, IP address based rate limit of 10 pulls per hour per address. We will surely have very infrequent success at best if we try to rely on the address based limits inside the GitHub Actions runner pool.

There are other, more drastic ways of skirting these limits (like not using Docker at all) but they are probably not worth jumping into without more detailed investigation.

(*)NOTE: I don't know what kind of Docker or Docker Hub account we have nor what subscriptions are attached to it, or if we even have such a thing already.

expanded error text:

from https://github.com/cmu-delphi/delphi-epidata/actions/runs/13040470808/job/36380975909?pr=1589#step:12:1561

#1 [internal] load build definition from Dockerfile
#1 transferring dockerfile: 156B done
#1 DONE 0.0s

#2 [internal] load metadata for docker.io/library/redis:latest
#2 ERROR: failed to copy: httpReadSeeker: failed open: unexpected status code https://registry-1.docker.io/v2/library/redis/manifests/sha256:ca65ea36ae16e709b0f1c7534bc7e5b5ac2e5bb3c97236e4fec00e3625eb678d: 429 Too Many Requests - Server message: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit
------
 > [internal] load metadata for docker.io/library/redis:latest:
------
Dockerfile:1
--------------------
   1 | >>> FROM redis
   2 |     
   3 |     ENV REDIS_PASSWORD=$REDIS_PASSWORD
--------------------
ERROR: failed to solve: redis: failed to resolve source metadata for docker.io/library/redis:latest: failed to copy: httpReadSeeker: failed open: unexpected status code https://registry-1.docker.io/v2/library/redis/manifests/sha256:ca65ea36ae16e709b0f1c7534bc7e5b5ac2e5bb3c97236e4fec00e3625eb678d: 429 Too Many Requests - Server message: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit
Error: Process completed with exit code 1.
@melange396 melange396 added chore devops building, running, deploying, environment stuff, handy utils, repository-related, engineer QoL, etc tests integration & unit testing, bug triage and prevention labels Jan 30, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
chore devops building, running, deploying, environment stuff, handy utils, repository-related, engineer QoL, etc tests integration & unit testing, bug triage and prevention
Projects
None yet
Development

No branches or pull requests

1 participant