calibrated-agent-evaluation capture¶
run an instrumented in-process agent, capture its OTel trace, and ingest it
usage: calibrated-agent-evaluation capture [-h] --agent AGENT [--service SERVICE]
[--trace-id TRACE_ID] [--dry-run] [--require-owned]
[--overwrite] [--backend {local,langfuse}]
[--root PATH] [--explain]
options:
-h, --help show this help message and exit
--agent AGENT the agent to run, as 'module:callable' (an instrumented, no-argument
entrypoint); the module is imported with the current directory first on
the path, as `python -m` does
--service SERVICE OTel service.name for the capture
--trace-id TRACE_ID trace id for the captured trace (default: the captured OTel trace id)
--dry-run capture and report; write nothing to the backend
--require-owned refuse the captured trace if it degrades to outcome-only
(ownership='probed')
--overwrite replace an existing trace with the same id (default: refuse, since a
capture cannot be re-run to recover what was replaced)
--backend {local,langfuse}
trace-store backend (default: local, offline)
--root PATH local root for the trace store and the stage records (default:
$CALIBRATED_AGENT_EVALUATION_LOCAL_ROOT, else ./.calibrated-agent-
evaluation; quickstart defaults to a temp dir)
--explain after the plain summary, print the statistics behind it (bounds, methods,
thresholds, record paths); CALIBRATED_AGENT_EVALUATION_EXPLAIN=1 turns it
on for every command