Failed to fetch all projects visible in https://xxxx.com. Query timeout was reached

[j14ncn]

The gitlab version has been upgraded to 16.3.9, and sourcebot can't work.

[backend] | 2025-01-15T05:51:09.740Z error: [GitLab] Failed to fetch all projects visible in https://git.xxx.com. Query timeout was reached
[backend] | GitbeakerTimeoutError: Query timeout was reached
[backend] |     at file:///app/node_modules/@gitbeaker/rest/dist/index.mjs:68:15
[backend] |     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
[backend] |     at async defaultRequestHandler (file:///app/node_modules/@gitbeaker/rest/dist/index.mjs:66:22)
[backend] |     at async getManyMore (file:///app/node_modules/@gitbeaker/core/dist/index.mjs:70:26)
[backend] |     at async measure (file:///app/packages/backend/dist/utils.js:5:18)
[backend] |     at async getGitLabReposFromConfig (file:///app/packages/backend/dist/gitlab.js:24:57)
[backend] |     at async syncConfig (file:///app/packages/backend/dist/main.js:193:37)
[backend] |     at async measure (file:///app/packages/backend/dist/utils.js:5:18)
[backend] |     at async _syncConfig (file:///app/packages/backend/dist/main.js:285:36)
[backend] |     at async main (file:///app/packages/backend/dist/main.js:319:5)

[brendan-kellam]

Looks like you hit a timeout - Are you also using a token? If not, try using a token since the allowed number of requests will be larger

[j14ncn]

yes ,We using token. Use the curl command can get the correct data.

[brendan-kellam]

How many repositories are in your GitLab install? I was mistaken, it's not rate limiting it's a query timeout

[j14ncn]

About 6,300 repos. Can we modify the query timeout?

[j14ncn]

[backend] |     at async _syncConfig (file:///app/packages/backend/dist/main.js:285:36)
[backend] |     at async main (file:///app/packages/backend/dist/main.js:319:5)
[backend] | 2025-01-15T06:06:53.936Z info: [main] Discovered 0 unique repositories from config.
[backend] | 2025-01-15T06:06:53.937Z info: [main] Synced configuration file /data/config.json in 300.051s

[brendan-kellam]

Looking at the gitbeaker docs, it defaults to 5 minutes, but can be modified via the queryTimeout param in the Gitlab client.

I think we can bump this to something like 30 minutes, which should be sufficient for use cases like yours with a ton of repos.

I can add that tomorrow, or feel free to open up a PR in the meantime 🤠

[j14ncn]

Maybe it's not a query timeout issue, I have another gitlab instance with only 10 repos and it still has a query timeout error.

[brendan-kellam]

Are you hitting the query timeout error with curl?

[j14ncn]

time curl -X GET "https://xxx.cn/api/v4/projects?per_page=1000" \ -H "Private-Token: tokens"|jq

0.08s user 0.03s system 1% cpu 6.210 total

[j14ncn]

We modify packages/backend/src/gitlab.ts:21 queryTimeout: 600000 and it seems to be working fine.