[Bug]: Rate limit Timeout Sleep WAY too long

[Kking112]

Version

Command-line (Python) version

Operating System

Ubuntu Linux

What happened?

When GPT Pilot runs into openai.RateLimitError, it will sleep for well over what is necessary based on the response of the API..

openai.RateLimitError: Error code: 429 - {'error': {'message': 'Rate limit reached for gpt-4o in {REMOVED FOR BUG REPORT} on tokens per min (TPM): Limit 30000, Used 29328, Requested 6310. Please try again in 11.276s. Visit https://platform.openai.com/account/rate-limits to learn more.', 'type': 'tokens', 'param': None, 'code': 'rate_limit_exceeded'}}

GPT Pilot should ideally sleep for the suggested time in the response of 11.276s (maybe add 1 second). Instead, it sleeps for either 7200 seconds or 9600 seconds.. sometimes it prints both. And I have let it sit for hours and it never seems to start up again.

I dug into the source code and I think I may be able to fix this, and if its necessary I will create a pull request to merge into the main repository, but I wanted to first make sure that I am not missing something and just need to change the config.json, or something similar.

Thank you in advance, and if you have had this issue too and there is no obvious fix, please let me know and I will start to fix it myself and create a pull request. I'd love to contribute to this awesome project!

[keshav2800]

hi, did you figured out how this will be solved?
i am facing the same issue

[losogudok]

I have same issue with Claude.
We've hit anthropic rate limit. Sleeping for 0 seconds...We've hit anthropic rate limit. Sleeping for 0 seconds...We've hit anthropic rate limit. Sleeping for 0 seconds...We've hit anthropic rate limit. Sleeping for 0 seconds...We've hit anthropic rate limit. Sleeping for 86399 seconds...
Is it possible to continue where you left off?

[mikerz]

Pretty significant issue, as soon as you hit rate limits gptpilot is bricked

[Bhalajee]

just replace this part of code :(find this in core.llm.base.py)

def rate_limit_sleep(self, err: Exception) -> Optional[datetime.timedelta]:
'''
Return how long we need to sleep because of rate limiting.
These are computed from the response headers that each LLM returns.
For details, check the implementation for the specific LLM. If there
are no rate limiting headers, we assume that the request should not
be retried and return None (this will be the case for insufficient
quota/funds in the account).
:param err: RateLimitError that was raised by the LLM client.
:return: optional timedelta to wait before trying again
'''
raise NotImplementedError()

with

import re
import datetime

def rate_limit_sleep(self, err: Exception) -> Optional[datetime.timedelta]:
# Extract the suggested wait time from the error message
match = re.search(r'Please try again in (\d+(.\d+)?)s', str(err))
if match:
wait_time = float(match.group(1))
# Return the wait time as a timedelta
return datetime.timedelta(seconds=wait_time + 1) # Add 1 second buffer
return None

[keshav2800]

@Bhalajee it didnot worked for me still its showing the same thing We've hit openai rate limit. Sleeping for 7200 seconds...