Skip to content

calibrated-agent-evaluation capture

run an instrumented in-process agent, capture its OTel trace, and ingest it

usage: calibrated-agent-evaluation capture [-h] --agent AGENT [--service SERVICE]
                                           [--trace-id TRACE_ID] [--dry-run] [--require-owned]
                                           [--overwrite] [--backend {local,langfuse}]
                                           [--root PATH] [--explain]

options:
  -h, --help            show this help message and exit
  --agent AGENT         the agent to run, as 'module:callable' (an instrumented, no-argument
                        entrypoint); the module is imported with the current directory first on
                        the path, as `python -m` does
  --service SERVICE     OTel service.name for the capture
  --trace-id TRACE_ID   trace id for the captured trace (default: the captured OTel trace id)
  --dry-run             capture and report; write nothing to the backend
  --require-owned       refuse the captured trace if it degrades to outcome-only
                        (ownership='probed')
  --overwrite           replace an existing trace with the same id (default: refuse, since a
                        capture cannot be re-run to recover what was replaced)
  --backend {local,langfuse}
                        trace-store backend (default: local, offline)
  --root PATH           local root for the trace store and the stage records (default:
                        $CALIBRATED_AGENT_EVALUATION_LOCAL_ROOT, else ./.calibrated-agent-
                        evaluation; quickstart defaults to a temp dir)
  --explain             after the plain summary, print the statistics behind it (bounds, methods,
                        thresholds, record paths); CALIBRATED_AGENT_EVALUATION_EXPLAIN=1 turns it
                        on for every command