日本語版はこちら

DurableFunctions template - Orchestrator-workers Multi-Agent

This repository is a template project for implementing the Orchestrator-Workers pattern introduced in Anthropic's blog "Building effective agents" using Azure Durable Functions.

ref: Anthropic-Building effective agents

Sequence Diagram

Case of synchronous endpoint:

sequenceDiagram
    participant Client
    participant Sarter
    participant DurableOrchestrator
    participant AgentDeciderActivity
    participant WorkerAgentActivity
    participant SynthesizerActivity

    Client->>Sarter: Web API request
    Sarter->>DurableOrchestrator: Start 
    DurableOrchestrator ->> AgentDeciderActivity: Decide agent to call
    AgentDeciderActivity ->> DurableOrchestrator: Agent that should call
    DurableOrchestrator -->> Sarter: If no agent to call, return plain text
    DurableOrchestrator ->> WorkerAgentActivity: Invoke(Multiple & Parallel)
    WorkerAgentActivity ->> DurableOrchestrator: Result
    DurableOrchestrator ->> SynthesizerActivity: Synthesize result & generate answer
    SynthesizerActivity ->> DurableOrchestrator: Synthesized answer
    DurableOrchestrator ->> Sarter: Answer
    Sarter ->> Client: Web API respons

Loading

Agent sample

This Multi-Agent system is based on a travel concierge scenario. Each Agent is set to return fixed values as responses for sample purposes. The sample Agents defined in the template are as follows:

• GetDestinationSuggestAgent：Get the destination suggestion
• GetClimateAgent：Get the climate of the destination
• GetSightseeingSpotAgent：Get the sightseeing spot of the destination
• GetHotelAgent：Get the hotel information of the destination
• SubmitReservationAgent：Submit the reservation of the hotel

Please modify the implementation of each agent, which currently uses fixed values, to include RAG, actions, and other processes, thereby creating agents that meet the requirements. Each agent can utilize OpenAI Client and application configuration values provided by the DI container.

To demonstrate the retry functionality of Durable Functions, each Agent includes code that emulates failures in external service calls, such as those to an LLM. Agent Activities fail randomly with a 30% probability during execution. When implementing an actual Agent based on the sample Agent, please remove the following parts from the code.

if(Random.Shared.Next(0, 10) < 3)
{
	logger.LogInformation("Failed to get climate information");
	throw new InvalidOperationException("Failed to get climate information");
}

Client to test

You can use client.py to test the Orchestrator-Workers pattern. This client made with Streamlit. So you can run it with the following command:

streamlit run client.py

The full-resolution video is here.

API

Endpoints

There are two types of endpoints: synchronous and asynchronous. If the agent's processing takes a long time, it is recommended to use the asynchronous endpoint. For more information about the asynchronous pattern in Durable Functions, see here.

• synchronoushttp://{your base url}/api/invoke/sync
• asynchronoushttp://{your base url}/api/invoke/async

API　request

The request body will be below:

{
	"messages": [
		{
			"role": "user",
			"content": "I'm finding a travel destination."
		},
		{
			"role": "assistant",
			"content": "Please tell me your desired conditions for the travel destination. For example, a place with a beach, a place with many historical tourist spots, a place with abundant nature, etc. I will suggest a travel destination according to your preferences."
		},
		{
			"role": "user",
			"content": "I like a place with many historical tourist spots."
		}
	],
	"requireAdditionalInfo": true
}

Response

The response will be below:

{
	"additionalInfo": [
		{
			"$type": "mardown",
			"markdownText": "### Recommended Travel Destinations with Historical Landmarks\n#### Domestic\n1. **Okinawa Main Island**\n  - Rich in tourist attractions such as crystal-clear beaches, Shurijo Castle, and the Churaumi Aquarium. \n- Warm weather even in winter, offering a relaxed atmosphere.\n2. **Ishigaki Island and Miyako Island**\n - Expansive beautiful natural landscapes typical of the tropics, with popular activities like diving and snorkeling.\n- Enjoy unique local cuisine of the islands.\n3. **Kagoshima and Amami Oshima**\n - Experience Amami's black sugar shochu, island songs, and distinctive natural environments.\n- Enjoy the subtropical atmosphere."
		}
	],
	"content": "Here are some recommended travel destinations with many historical landmarks:\n1. **Okinawa Main Island** - Features spots that blend history and tourism, such as Shurijo Castle and the Churaumi Aquarium.\n2. **Kagoshima and Amami Oshima** - Enjoy island songs and the subtropical atmosphere.\n3. **Ishigaki Island and Miyako Island** - Known for unique local cuisine and natural landscapes.\nFor more details, please refer to the additional information.",
	"calledAgentNames": [
		"GetDestinationSuggestAgent"
	]
}

Request for additional information

In general, responses from agents, especially when they include retrieval-augmented generation (RAG), can become lengthy and negatively affect the chat experience. To improve this, separating static additional information from the flow of the chat can enhance the user experience.

In this template, you can request additional information by setting requireAdditionalInfo to true when making a request. The additional information will be returned separately from the chat response and stored in the additionalInfo field.

By toggling the REQUIRE_ADDITIONAL_INFO flag in the client code for testing, you can experience this functionality in action.

This feature also reduces the token count of the messages array in the chat history, making the LLM's performance lighter. However, in some cases, the lack of context in the messages array might result in unnatural agent responses. In such cases, you may consider merging the additional information back into the messages array and requesting the agent's response accordingly.