Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AzureOpenAiChatModel Retry Mechanism Not Working. #2055

Open
shelltea opened this issue Jan 8, 2025 · 0 comments
Open

AzureOpenAiChatModel Retry Mechanism Not Working. #2055

shelltea opened this issue Jan 8, 2025 · 0 comments

Comments

@shelltea
Copy link

shelltea commented Jan 8, 2025

Bug description
The AzureOpenAiChatModel does not support the retry mechanism. According to the documentation, this feature should already be supported. However, after configuring the following parameters, retries still do not occur:

spring.ai.retry.on-client-errors=true
spring.ai.retry.on-http-codes=400,408,429,500,502,503,504

Environment

  • Spring AI version: 1.0.0-M5
  • Java version: 21

Steps to reproduce

  1. Configure the retry settings in your application properties file as follows:
    spring.ai.retry.on-client-errors=true
    spring.ai.retry.on-http-codes=400,408,429,500,502,503,504
  2. Invoke the AzureOpenAiChatModel under conditions that would trigger a retry (e.g., by forcing an HTTP 400 or other configured codes).
  3. Observe that no retry occurs.

Expected behavior
The AzureOpenAiChatModel should attempt to retry the request according to the specified retry configuration parameters.

Error Logs

com.azure.core.exception.HttpResponseException: Status code 400, "{"error":{"inner_error":{"code":"ResponsibleAIPolicyViolation","content_filter_results":{"sexual":{"filtered":true,"severity":"high"},"violence":{"filtered":false,"severity":"safe"},"hate":{"filtered":false,"severity":"safe"},"self_harm":{"filtered":false,"severity":"safe"}}},"code":"content_filter","message":"The response was filtered due to the prompt triggering Azure OpenAI's content management policy. Please modify your prompt and retry. To learn more about our content filtering policies please read our documentation: \r\nhttps://go.microsoft.com/fwlink/?linkid=2198766.","param":"prompt","type":null}}"

	at com.azure.core.implementation.http.rest.RestProxyBase.instantiateUnexpectedException(RestProxyBase.java:388)
	at com.azure.core.implementation.http.rest.SyncRestProxy.ensureExpectedStatus(SyncRestProxy.java:133)
	at com.azure.core.implementation.http.rest.SyncRestProxy.handleRestReturnType(SyncRestProxy.java:211)
	at com.azure.core.implementation.http.rest.SyncRestProxy.invoke(SyncRestProxy.java:86)
	at com.azure.core.implementation.http.rest.RestProxyBase.invoke(RestProxyBase.java:124)
	at com.azure.core.http.rest.RestProxy.invoke(RestProxy.java:95)
	at jdk.proxy2/jdk.proxy2.$Proxy196.getChatCompletionsSync(Unknown Source)
	at com.azure.ai.openai.implementation.OpenAIClientImpl.getChatCompletionsWithResponse(OpenAIClientImpl.java:1900)
	at com.azure.ai.openai.OpenAIClient.getChatCompletionsWithResponse(OpenAIClient.java:350)
	at com.azure.ai.openai.OpenAIClient.getChatCompletions(OpenAIClient.java:760)
	at org.springframework.ai.azure.openai.AzureOpenAiChatModel.lambda$internalCall$1(AzureOpenAiChatModel.java:244)
	at io.micrometer.observation.Observation.observe(Observation.java:565)
	at org.springframework.ai.azure.openai.AzureOpenAiChatModel.internalCall(AzureOpenAiChatModel.java:240)
	at org.springframework.ai.azure.openai.AzureOpenAiChatModel.call(AzureOpenAiChatModel.java:226)
	at org.springframework.ai.chat.client.DefaultChatClient$DefaultChatClientRequestSpec$1.aroundCall(DefaultChatClient.java:675)
	at org.springframework.ai.chat.client.advisor.DefaultAroundAdvisorChain.lambda$nextAroundCall$1(DefaultAroundAdvisorChain.java:98)
	at io.micrometer.observation.Observation.observe(Observation.java:565)
	at org.springframework.ai.chat.client.advisor.DefaultAroundAdvisorChain.nextAroundCall(DefaultAroundAdvisorChain.java:98)
	at org.springframework.ai.chat.client.DefaultChatClient$DefaultCallResponseSpec.doGetChatResponse(DefaultChatClient.java:488)
	at org.springframework.ai.chat.client.DefaultChatClient$DefaultCallResponseSpec.lambda$doGetObservableChatResponse$1(DefaultChatClient.java:477)
	at io.micrometer.observation.Observation.observe(Observation.java:565)
	at org.springframework.ai.chat.client.DefaultChatClient$DefaultCallResponseSpec.doGetObservableChatResponse(DefaultChatClient.java:477)
	at org.springframework.ai.chat.client.DefaultChatClient$DefaultCallResponseSpec.doSingleWithBeanOutputConverter(DefaultChatClient.java:451)
	at org.springframework.ai.chat.client.DefaultChatClient$DefaultCallResponseSpec.entity(DefaultChatClient.java:446)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants
@shelltea and others