Overview

RL on agents is different from RL on text completion. Rollouts are long-horizon, multi-turn, and tool-using — episodes run for minutes / hours hitting shells, APIs, or other tools. The hard parts are operational: running hundreds in isolation in parallel, capturing the signals a trainer needs (token IDs, logprobs, rewards), and doing it without rewiring the agent for RL.

The AgentCore RL Toolkit’s goal: the agent you train is the agent you deploy. Keep your Bedrock AgentCore agent and connect it to a training backend of your choice.

Training Backends

Native Integrations

Training integrations maintained in this repository use ART’s rollout gateway for token-level trajectory capture.

Integration	Description	Get Started
verl	Self-hosted training with verl	Setup guide
slime	Self-hosted training with slime	Setup guide
SageMaker	Managed training on Amazon SageMaker Training Sessions (experimental)	Setup guide
SkyRL	Self-hosted training through its Tinker-compatible API	Training guide · Deployment example
Tinker	Managed training through the official Tinker API	Training guide

SkyRL and Tinker share the toolkit’s Tinker-compatible API integration.

External Integrations

rLLM natively integrates with AWS Bedrock AgentCore Runtime. Follow the rLLM documentation for installation, configuration, and training. Use ART to prepare and deploy your agent.

Why AgentCore Runtime

AgentCore Runtime is AWS’s serverless hosting layer for agents (see the developer guide for broader AgentCore features). For RL, four properties matter, all available in AgentCore Runtime without bespoke infrastructure:

• Session isolation — each session runs in a dedicated microVM with isolated CPU, memory, and filesystem, giving complete separation between user sessions and preventing cross-session data contamination. Filesystem writes in one rollout are invisible to any other. After session completion the microVM is terminated and memory sanitized.
• On-demand scaling — the runtime is serverless; new sessions spin up on demand, enabling massive parallel rollouts without contending for local CPU or reserving worker capacity.
• Extended execution time — real-time interactions and long-running workloads up to 8 hours — wide enough for multi-turn tool-using rollouts.
• Framework agnostic — any agent framework (LangGraph, Strands, or custom) and any foundation model. The agent’s runtime environment is independent of the training library’s, so the two evolve separately.

How the toolkit works

AgentCore Runtime handles hosting; the toolkit wires the training-data pipeline that makes an agent trainable.

The diagram has three parts, left to right:

• AgentCore Runtime (left) — where your agent runs. Each rollout is a session with an Agent ↔ Environment tool-use loop that ends in an Evaluate step. Your code lives here, unchanged from deployment.
• Rollout gateway (center) — ART’s RolloutGateway accepts standard OpenAI- and Anthropic-compatible chat requests. It renders messages into prompt token IDs, samples through a token-in/token-out backend, and records generated token IDs and logprobs. A session key in the API-key field groups calls into trajectories with loss masks, preserving the original generated tokens across turns and branches. The trainer joins these trajectories with the agent’s rewards.
• Training backend (right) — Inference Servers serve the current policy; the Training Engine consumes rollout data + rewards and syncs updated weights back via Weight Sync.

Further reading

Our blog post Training multi-step agents on AgentCore Runtime with reinforcement learning walks through this system end-to-end and shares results from training agents on two long-horizon benchmarks — MigrationBench (Java code-migration) and OfficeBench (office automation). RL lifts task success from 43% → 76% on MigrationBench and roughly doubles success on OfficeBench, with the largest gains on the hardest multi-app tasks.

Next

Choose a guide from Training Backends for agent preparation and training setup.