Skip to content
Multi-Agent Orchestrator

FastAPI Streaming

This example demonstrates how to implement streaming responses with the Multi-Agent Orchestrator using FastAPI. It shows how to build a simple API that streams responses from multiple AI agents in real-time.

Features

  • Real-time streaming responses using FastAPI’s StreamingResponse
  • Custom streaming handler implementation
  • Multiple agent support (Tech and Health agents)
  • Queue-based token streaming
  • CORS-enabled API endpoint

Quick Start

Terminal window
# Install dependencies
pip install "fastapi[all]" multi-agent-orchestrator


# Run the server
uvicorn app:app --reload

API Endpoint

Terminal window
POST /stream_chat/

Request body:

{
    "content": "your question here",
    "user_id": "user123",
    "session_id": "session456"
}

Implementation Highlights

  • Uses FastAPI’s event streaming capabilities
  • Custom callback handler for real-time token streaming
  • Thread-safe queue implementation for token management
  • Configurable orchestrator with multiple specialized agents
  • Error handling and proper stream closure

Example Usage

import requests


response = requests.post(
    'http://localhost:8000/stream_chat/',
    json={
        'content': 'What are the latest AI trends?',
        'user_id': 'user123',
        'session_id': 'session456'
    },
    stream=True
)


for chunk in response.iter_content():
    print(chunk.decode(), end='', flush=True)

Ready to build your own multi-agent chat application? Check out the complete source code in our GitHub repository.