Refact AdvisedRequest to avoid userText rendering

[ghdcksgml1]

Hello.

While implementing RAG using spring-ai, I encountered an error during rendering with PromptTemplate when the userText contained curly braces ({}).

// My Request
{
    "prompt": "\"``` \\n suspend fun methodName(prompt: String) { \\n     return RetryUtil.retry( \\n         context = Dispatchers.IO, \\n         retryCount = 3, \\n         predicate = { \\n             logger.info(\\\"RETRY POLICY :: ${it.message}\\\") \\n             return@retry it is RuntimeException \\n         } \\n     ) { \\n         methodGeneratorClient.prompt() \\n             .user(prompt) \\n             .advisors() \\n             .call() \\n             .entity(object : ParameterizedTypeReference<List<MethodGenerateResult>>() {}) ?: emptyList() \\n     } \\n } \\n ``` \\n\\nPlease Recommend Method name\""
}

Configuration

Spring Boot : 3.4.1
jdk : 17
spring-ai : 1.0.0-M5

private var SYSTEM_MESSAGE: String = """
    You are an expert in defining method names by analyzing code. 
    Provide clear and concise method names that match the functionality and purpose of the code snippets provided by the user. 
    Method names should start with a verb, followed by an object or adjective to clearly convey the meaning. 
    If the user requests names for multiple methods, separate each name with a newline. 
    Do not provide unnecessary explanations.
""".trimIndent()

@Configuration
class MethodGeneratorPipeline {

    @Bean
    fun methodGeneratorClient(
        vectorStore: VectorStore,
        ollamaChatModel: OllamaChatModel,
    ): ChatClient {
        return ChatClient.builder(ollamaChatModel)
            .defaultSystem(SYSTEM_MESSAGE)
            .defaultAdvisors(methodGeneratorPipeline(vectorStore))
            .build()
    }

    private fun methodGeneratorPipeline(vectorStore: VectorStore): BaseAdvisor {
        val retriever = VectorStoreDocumentRetriever.builder()
            .vectorStore(vectorStore)
            .similarityThreshold(0.73)
            .topK(5)
            .build()

        val queryAugmenter = ContextualQueryAugmenter.builder()
            .allowEmptyContext(true)
            .build()

        return RetrievalAugmentationAdvisor.builder()
            .documentRetriever(retriever)
            .documentJoiner(ConcatenationDocumentJoiner())
            .queryAugmenter(queryAugmenter)
            .build()
    }
}

@RestController
@RequestMapping("/methods")
class MethodController(
    private val methodGeneratorClient: ChatClient,
) {
    private val logger = logger<MethodController>()

    @PostMapping(value = [""], produces = [MediaType.APPLICATION_JSON_VALUE])
    suspend fun generateMethodNames(
        @RequestBody request: MethodGenerateRequest,
    ): List<MethodGenerateResult> {

        return retry(
            context = Dispatchers.IO,
            retryCount = 1,
            predicate = {
                logger.info("RETRY POLICY :: ${it.message}")
                return@retry it is RuntimeException
            }
        ) {
            methodGeneratorClient.prompt()
                .user(request.prompt)
                .call()
                .entity(object : ParameterizedTypeReference<List<MethodGenerateResult>>() {}) ?: emptyList()
        }
    }
}

== error log ==
1:49: invalid character '\'
1:56: 'return' came as a complete surprise to me

// Json Response
{
    "error": {
        "status": "400",
        "title": "ERROR",
        "message": "The template string is not valid."
    }
}

Looking through the history, I found many related issues concerning StringTemplate. #355

When I implemented it using custom templates without the RetrievalAugmentationAdvisor, the error did not occur. However, the error started appearing after using RetrievalAugmentationAdvisor.

After investigating the cause, I discovered that the Advisor was treating userText as if it were a template and rendering it.

Query originalQuery = new Query(new PromptTemplate(request.userText(), request.userParams()).render());

Therefore, I propose a solution to manage the text generated by the Advisor separately as a variable called advisorText instead of replacing the userText.

userText should provide the rendered string only when direct rendering is required, such as in DocumentRetrieval or ChatMemory.

I envisioned a scenario where the actual rendering of userText happens in the toPrompt() method of CallAroundAdvisor, which is created by default in DefaultChatClientRequestSpec.

		public DefaultChatClientRequestSpec(...) {
                         ...
			this.advisors.add(new CallAroundAdvisor() {

				@Override
				public String getName() {
					return CallAroundAdvisor.class.getSimpleName();
				}

				@Override
				public int getOrder() {
					return Ordered.LOWEST_PRECEDENCE;
				}

				@Override
				public AdvisedResponse aroundCall(AdvisedRequest advisedRequest, CallAroundAdvisorChain chain) {
					return new AdvisedResponse(chatModel.call(advisedRequest.toPrompt()), Collections.unmodifiableMap(advisedRequest.adviseContext()));
				}
			});
                         ...
		}