Skip to main content

2 posts tagged with "Deep dive"

How something works under the hood, and why it was built that way.

View All Tags

Why physical AI needs multi-agent orchestration

· 35 min read

Robots now work in mixed teams: each area of a factory, a warehouse, or a laboratory has its own robot team. When an item crosses from one area to the next, the hard part is the coordination, not the motion. Someone must plan the route, pass the responsibility for the item to the next team, and confirm that the job is done. Each site builds this coordination itself, and the work grows with the fleets.

Agentic AI can take on this coordination. Agentic AI is AI that works toward a goal on its own. An AI agent uses a large language model (LLM), the type of model behind chat assistants, to read a goal in natural language. Then it plans the steps, and calls software tools that carry them out. But an agent that moves machines needs limits that a chat assistant does not need. It must act only within the authority of its team. And the proof that a step is done must come from the physical world, not from the text of another agent.

Multi-agent orchestration is the coordination of several AI agents, each with its own role, so that together they do a job that no single agent should do alone. CLI Agent Orchestrator (CAO) is an open-source tool for multi-agent orchestration. It runs a team of CLI agents: AI assistants that work in a terminal, for example Claude Code, Kiro CLI, or Codex. Each agent has its own role, its own tools, and its own credential, the key that sets what the agent can do.

In this post, you learn why physical AI needs agentic AI, and why agentic AI that moves machines needs orchestration. The first physical AI example of CAO, cross-zone transport, applies CAO to a parcel on a simulated factory floor. A supervisor agent plans the job. Two zone worker agents each move only the robots of their own zone, and an independent checker agent verifies the result. In 10 live runs, every run ended in the expected state. Custody is the responsibility for the parcel. Exactly one zone has custody at a time, so one team at a time is responsible for the parcel. Custody changed 6 times, and each time only after the simulator measured the parcel at the dock. The 3 requests that were not possible ended with no command to any robot. You also see the five criteria that a team of agents must meet on a shared floor, and how the example meets each one with CAO. Then you see what a run looks like, and how the example maps to a real site.

The simulated floor at the end of the demo. The pink parcel is on the green east cart at etch, in the east zone. The blue west cart waits at the amber dock.

Play the animation of the demo (13 seconds, repeats)

Animation of the cross-zone transport in MuJoCo. The blue west cart carries the pink parcel from stock to the amber dock. West offers custody to east, and east accepts it. Then the green east cart carries the parcel to etch, and the checker confirms the parcel at etch, owner east.

The demo: two zones. The animation replays the commands of a live run in the MuJoCo world of the example: the same robots, commands, and order. It leaves out the time that the agents use between the steps. The Results section shows an extended demo with three zones.

How CAO memory helps you and your agents keep context

· 17 min read

With Agentic context engineering, how an agent uses memory is critical in many aspects. It helps you increase code quality, be more token efficient, and keep consistency across a long-running session or session handoff.

Memory is not just giving the next session the conclusion or summary of the previous one. CAO memory saves conclusions that make the next task faster. It is more than conversation history. It can hold project decisions, user preferences, reusable instructions, document findings, and workflow lessons.

The result is closer to a scoped project wiki than a transcript archive. The identity of CAO memory is human readable markdown files. Search runs over an index backed by a SQLite database, and a knowledge graph projects the relationships between memory nodes, both in the CAO UI and in Obsidian.

CAO gives every supported agent the same memory tools. An agent can remember a fact with one CLI provider and recall it later with another. The memory belongs to CAO, not to a specific model or CLI, so your context (and code quality) survives handoffs.

This post explains how that shared layer works. For commands and configuration, see the original CAO memory reference.