Orchestrator overview
The Multi-Agent Orchestrator is the central component of the framework, responsible for managing agents, routing requests, and handling conversations. This page provides an overview of how to initialize the Orchestrator and details all available configuration options.
Initializing the Orchestrator
To create a new Orchestrator instance, you can use the MultiAgentOrchestrator
class:
The options
parameter is optional and allows you to customize various aspects of the Orchestrator’s behavior.
Configuration options
The Orchestrator accepts an OrchestratorConfig
object during initialization. All options are optional and will use default values if not specified. Here’s a complete list of available options:
storage
: Specifies the storage mechanism for chat history. Default isInMemoryChatStorage
.config
: An instance ofOrchestratorConfig
containing various configuration flags and values:LOG_AGENT_CHAT
: Boolean flag to log agent chat interactions.LOG_CLASSIFIER_CHAT
: Boolean flag to log classifier chat interactions.LOG_CLASSIFIER_RAW_OUTPUT
: Boolean flag to log raw classifier output.LOG_CLASSIFIER_OUTPUT
: Boolean flag to log processed classifier output.LOG_EXECUTION_TIMES
: Boolean flag to log execution times of various operations.MAX_RETRIES
: Number of maximum retry attempts for the classifier.MAX_MESSAGE_PAIRS_PER_AGENT
: Maximum number of message pairs to retain per agent.USE_DEFAULT_AGENT_IF_NONE_IDENTIFIED
: Boolean flag to use the default agent when no specific agent is identified.CLASSIFICATION_ERROR_MESSAGE
: Custom error message for classification errors.NO_SELECTED_AGENT_MESSAGE
: Custom message when no agent is selected.GENERAL_ROUTING_ERROR_MSG_MESSAGE
: Custom message for general routing errors.
logger
: Custom logger instance. If not provided, a default logger will be used.classifier
: Custom classifier instance. If not provided, aBedrockClassifier
will be used.
Example with all options
Here’s an example that demonstrates how to initialize the Orchestrator with all available options:
Remember, all these options are optional. If you don’t specify an option, the Orchestrator will use its default value.
Default values
The default configuration is defined as follows:
In both implementations, DEFAULT_CONFIG
is an instance of OrchestratorConfig
with default values.
Available Functions
The MultiAgentOrchestrator provides several key functions to manage agents, process requests, and configure the orchestrator. Here’s a detailed overview of each function, explaining what it does and why you might use it:
Let’s break down each function:
-
addAgent (TypeScript) / add_agent (Python)
- What it does: Adds a new agent to the orchestrator.
- Why use it: Use this function to expand the capabilities of your system by introducing new specialized agents. Each agent can handle specific types of queries or tasks.
- Example use case: Adding a weather agent to handle weather-related queries, or a booking agent for reservation tasks.
-
getDefaultAgent
- What it does: Retrieves the current default agent.
- Why use it: This function is useful when you need to reference or use the default agent, perhaps for fallback scenarios or to compare its capabilities with other agents.
- Example use case: Checking the current default agent’s configuration before deciding whether to replace it.
-
setDefaultAgent
- What it does: Sets a new default agent for the orchestrator.
- Why use it: This allows you to change the fallback agent used when no specific agent is selected for a query. It’s useful for customizing the general-purpose response handling of your system.
- Example use case: Replacing the default generalist agent with a more specialized one that better fits your application’s primary use case.
-
setClassifier
- What it does: Sets a new classifier for the orchestrator.
- Why use it: This function allows you to change how the orchestrator determines which agent should handle each request. You might use it to implement a custom classification strategy or to use a different machine learning model for intent classification.
- Example use case: Implementing a domain-specific classifier that’s more accurate for your particular use case than the default classifier.
-
getAllAgents
- What it does: Retrieves a dictionary of all registered agents, including their names and descriptions.
- Why use it: This function is useful for getting an overview of all available agents in the system. It can be used for debugging, logging, or providing user-facing information about system capabilities.
- Example use case: Generating a help message that lists all available agents and their capabilities.
-
routeRequest
- What it does: This is the main function for processing user requests. It takes a user’s input, classifies it, selects an appropriate agent, and returns the agent’s response.
- Why use it: This is the core function you’ll use to handle user interactions in your application. It encapsulates the entire process of understanding the user’s intent and generating an appropriate response.
- Example use case: Processing a user’s message in a chatbot interface and returning the appropriate response.
Each of these functions plays a crucial role in configuring and operating the Multi-Agent Orchestrator. By using them effectively, you can create a flexible, powerful system capable of handling a wide range of user requests across multiple domains.
These functions allow you to configure the orchestrator, manage agents, and process user requests. For more detailed information on each function, please refer to the API Reference section.
Agent Selection and Default Behavior
When a user sends a request to the Multi-Agent Orchestrator, the system attempts to classify the intent and select an appropriate agent to handle the request. However, there are cases where no specific agent is selected.
When No Agent is Selected
If the classifier cannot confidently determine which agent should handle a request, it may result in no agent being selected. The orchestrator’s behavior in this case depends on the USE_DEFAULT_AGENT_IF_NONE_IDENTIFIED
configuration option:
-
If
USE_DEFAULT_AGENT_IF_NONE_IDENTIFIED
isTrue
(default):- The orchestrator will use the default agent to handle the request.
- This ensures that users always receive a response, even if it’s from a generalist agent.
-
If
USE_DEFAULT_AGENT_IF_NONE_IDENTIFIED
isFalse
:- The orchestrator will return a message specified by the
NO_SELECTED_AGENT_MESSAGE
configuration. - This prompts the user to rephrase their request for better agent identification.
- The orchestrator will return a message specified by the
Default Agent
The default agent is a BedrockLLMAgent
configured as a generalist, capable of handling a wide range of topics. It’s used when:
- No specific agent is selected and
USE_DEFAULT_AGENT_IF_NONE_IDENTIFIED
isTrue
. - You explicitly set it as the fallback option.
You can customize the default agent or replace it entirely using the set_default_agent
method:
Customizing NO_SELECTED_AGENT_MESSAGE
You can customize the message returned when no agent is selected (and USE_DEFAULT_AGENT_IF_NONE_IDENTIFIED
is False
) by setting the NO_SELECTED_AGENT_MESSAGE
in the orchestrator configuration:
Best Practices
-
Default Agent Usage: Use the default agent when you want to ensure all user queries receive a response, even if it’s not from a specialized agent.
-
Prompting for Clarification: Set
USE_DEFAULT_AGENT_IF_NONE_IDENTIFIED
toFalse
and customize theNO_SELECTED_AGENT_MESSAGE
when you want to encourage users to provide more specific or clear requests. -
Balancing Specificity and Coverage: Consider your use case carefully. Using a default agent provides broader coverage but may sacrifice specificity. Prompting for clarification may lead to more accurate agent selection but requires additional user interaction.
-
Monitoring and Iteration: Regularly review cases where no agent is selected. This can help you identify gaps in your agent coverage or refine your classification process.
By understanding and customizing these behaviors, you can fine-tune your Multi-Agent Orchestrator to provide the best possible user experience for your specific use case.
Additional notes
-
The
storage
option allows you to specify a custom storage mechanism. By default, it uses in-memory storage (InMemoryChatStorage
), but you can implement your own storage solution or use built-in options like DynamoDB storage. For more information, see the Storage section. -
The
logger
option lets you provide a custom logger. If not specified, a default logger will be used. To learn how to implement a custom logger, check out the logging section. -
The
classifier
option allows you to use a custom classifier for intent classification. If not provided, aBedrockClassifier
will be used by default. For details on implementing a custom classifier, see the Custom Classifiers documentation.
By customizing these options, you can tailor the Orchestrator’s behavior to suit your specific use case and requirements.