Skip to content

calibrated_agent_evaluation.review.renderers

calibrated_agent_evaluation.review.renderers

Plugin-renderer seam for the review surface.

RendererRegistry (in base) is the FastHTML-free resolver; tool_call_tree_renderer (in default) is the built-in fallback view. make_default_registry wires them together: the app builds one, and a domain registers its own renderers on it via register(predicate, fn). The concrete renderers need fasthtml (the [ui] extra), so they load on first use. Importing this package for RendererRegistry alone, as calibrated_agent_evaluation.review does, needs no extra.

Renderer module-attribute

Renderer = Callable[[Trace], Any]

RendererPredicate module-attribute

RendererPredicate = Callable[[Trace], bool]

RendererRegistry dataclass

RendererRegistry(default, _rules=list())

Resolve a trace's renderer: the first registered predicate that matches, else the default.

Construct with the default renderer (the app wires the tool-call-tree renderer). Register domain renderers with register. render is the safe entry point; it guards a raising renderer back to the default and logs, so one bad plugin never ends the session.

default instance-attribute

default

register

register(predicate, renderer)

Register renderer for any trace where predicate(trace) is true.

Registration order is match order; the first matching predicate wins.

Source code in src/calibrated_agent_evaluation/review/renderers/base.py
def register(self, predicate: RendererPredicate, renderer: Renderer) -> None:
    """Register ``renderer`` for any trace where ``predicate(trace)`` is true.

    Registration order is match order; the first matching predicate wins.
    """
    self._rules.append((predicate, renderer))

resolve

resolve(trace)

Return the first renderer whose predicate matches trace, else the default.

Note: a predicate that itself raises is treated as non-matching (guarded + logged) so a broken predicate cannot mask the default.

Source code in src/calibrated_agent_evaluation/review/renderers/base.py
def resolve(self, trace: Trace) -> Renderer:
    """Return the first renderer whose predicate matches ``trace``, else the default.

    Note: a predicate that itself raises is treated as non-matching (guarded + logged) so a
    broken predicate cannot mask the default.
    """
    for predicate, renderer in self._rules:
        try:
            matched = predicate(trace)
        except Exception:
            logger.exception(
                "renderer predicate raised for trace %s; treating as no-match", trace.trace_id
            )
            continue
        if matched:
            return renderer
    return self.default

render

render(trace)

Render trace with the resolved renderer; fall back to the default on error.

A registered renderer that raises is caught and logged (logging.exception) and the default renderer is used instead; the labeling session continues rather than crashing. Use render_with_status to also learn that a fallback happened.

Source code in src/calibrated_agent_evaluation/review/renderers/base.py
def render(self, trace: Trace) -> Any:
    """Render ``trace`` with the resolved renderer; fall back to the default on error.

    A registered renderer that raises is caught and logged (``logging.exception``) and the
    default renderer is used instead; the labeling session continues rather than crashing.
    Use :meth:`render_with_status` to also learn that a fallback happened.
    """
    output, _fallback = self.render_with_status(trace)
    return output

render_with_status

render_with_status(trace)

Like render, plus a message naming the failure when the default stood in.

The message is for the labeler's screen. A fallback that only reached the log looked, to the person labeling, exactly like a trace the plugin was never meant to render.

Source code in src/calibrated_agent_evaluation/review/renderers/base.py
def render_with_status(self, trace: Trace) -> tuple[Any, str | None]:
    """Like :meth:`render`, plus a message naming the failure when the default stood in.

    The message is for the labeler's screen. A fallback that only reached the log looked, to
    the person labeling, exactly like a trace the plugin was never meant to render.
    """
    renderer = self.resolve(trace)
    try:
        return renderer(trace), None
    except Exception as exc:
        logger.exception(
            "plugin renderer failed for trace %s; falling back to the default renderer",
            trace.trace_id,
        )
        name = getattr(renderer, "__name__", type(renderer).__name__)
        message = (
            f"The {name!r} renderer for this trace failed ({type(exc).__name__}: {exc}); "
            "showing the default view instead."
        )
        return self.default(trace), message

tool_call_tree_renderer

tool_call_tree_renderer(trace)

Render trace as its terminal I/O plus (for owned traces) the tool-call span tree.

Source code in src/calibrated_agent_evaluation/review/renderers/default.py
def tool_call_tree_renderer(trace: Trace) -> Any:
    """Render ``trace`` as its terminal I/O plus (for owned traces) the tool-call span tree."""
    header = Div(
        H3(f"Trace {trace.trace_id}"),
        Span(
            f"frame={trace.frame} · ownership={trace.ownership} · "
            f"cell={_pretty(trace.taxonomy_cell)}",
            cls="hd-muted",
        ),
        cls="hd-trace-header",
    )
    terminal = Section(
        Div(Strong("input: "), _block(trace.input, "hd-io")),
        Div(Strong("output: "), _block(trace.output, "hd-io")),
        cls="hd-terminal",
    )
    if trace.spans:
        body = Section(H4("Span tree"), _span_tree(trace), cls="hd-spans")
    else:
        body = Section(
            Span("probed trace: no spans (outcome-only)", cls="hd-muted"), cls="hd-spans"
        )
    return Div(header, terminal, body, cls="hd-trace", data_trace_id=trace.trace_id)

make_default_registry

make_default_registry()

A fresh registry whose default is the built-in tool-call-tree renderer.

Source code in src/calibrated_agent_evaluation/review/renderers/__init__.py
def make_default_registry() -> RendererRegistry:
    """A fresh registry whose default is the built-in tool-call-tree renderer."""
    from calibrated_agent_evaluation.review.renderers.default import tool_call_tree_renderer

    return RendererRegistry(default=tool_call_tree_renderer)