Coverage for gco_mcp/mission/decide.py: 96.99%
87 statements
« prev ^ index » next coverage.py v7.15.2, created at 2026-07-30 21:22 +0000
1"""Pure deterministic verdict cascade for the Mission Decide_Phase.
3The cascade is the **control-path** output of the loop: given the current
4``SessionState`` (before the in-progress iteration is appended), the
5in-progress :class:`IterationRecord` (with ``strategy``, ``observation``,
6and ``criteria_evaluation`` already populated but ``verdict`` /
7``verdict_reason`` not yet set), and the wall-clock value the caller has
8already measured, :func:`decide_verdict` returns a
9``(VerdictLabel, VerdictReason)`` tuple. The function is pure: no logger
10calls, no I/O, no random sources, no clock reads. The wall-clock value is
11passed in on the call signature so tests can pin it.
13The cascade order is fixed:
151. **Budget terminations** — checked in a fixed sub-order so the
16 verdict_reason is deterministic when more than one cap is breached:
18 * ``max_iterations`` — the in-progress iteration would be the
19 ``budget["max_iterations"]``-th or later. Computed as
20 ``len(session["iterations"]) + 1 >= max_iterations``.
21 * ``max_wall_clock`` — ``now - session["started_at"] >= max_wall_clock_seconds``.
22 Returns False when ``started_at`` is missing (the session has
23 not yet transitioned out of ``pending``).
24 * ``no_progress`` — ``no_progress_counter >= stagnation_threshold``.
25 When the session has ``use_sampling=true``, the heuristic (step
26 4) gets priority so the sampler can revise the strategy before
27 the loop terminates. Without sampling, ``no_progress``
28 terminates immediately. If the heuristic doesn't fire (e.g.,
29 the tool sequence changed after a prior revision), the deferred
30 stagnation check (step 4b) terminates.
322. **Completion** — every ``required=True`` Criterion has status
33 ``met`` in the in-progress iteration's ``criteria_evaluation``, AND
34 no Criterion (required or not) has status ``inconclusive``.
363. **Cadence-skip** — when :func:`should_evaluate_now` says "this is
37 not a checkpoint", emit a synthetic ``("continue", "cadence_skip")``
38 without consulting the Strategy_Revision_Heuristic. The heuristic
39 only fires on real checkpoints so off-cadence iterations cannot
40 advance the no-progress counter or trigger an ``adjust``.
424. **Strategy_Revision_Heuristic** — :func:`_strategy_unproductive`:
43 the same ``tool_calls[*].tool_name`` sequence
44 for the last 3 iterations AND ``no_progress_counter`` at or above
45 half the stagnation threshold, OR new errors in the latest
46 Observation that didn't appear in the prior Observation. Returns
47 ``("adjust", "heuristic_unproductive")`` when either clause fires.
494b. **Deferred stagnation** — if step 1c deferred the ``no_progress``
50 check (because sampling is enabled) and the heuristic didn't fire,
51 terminate now.
535. **Default** — ``("continue", "in_progress")``.
55The ``iteration`` argument is *not* yet present in
56``session["iterations"]`` — the engine appends it after the verdict is
57decided. Anything that needs to look at "the last N iterations
58including the current one" composes the current ``iteration`` with
59``session["iterations"][-(N-1):]``.
61Determinism: same ``(session, iteration, now)`` triples produce the same
62``(VerdictLabel, VerdictReason)`` tuples. This is enforced by a property
63test in ``tests/test_mission_decide_determinism.py``.
65Cost guardrails are intentionally absent from this cascade. Real-time
66workload cost tracking is structurally inaccurate (Spot vs on-demand
67drift, EBS / EFA / egress not in the Pricing API, Cost Explorer 24h
68latency). Operators who need a cost cap should configure AWS Budgets
69and Cost Anomaly Detection at the account level — Mission caps only
70the controls the loop has direct visibility into.
71"""
73from __future__ import annotations
75import math
76from datetime import datetime, timedelta
78from .checkpoints import should_evaluate_now
79from .types import IterationRecord, SessionState, VerdictLabel, VerdictReason
81# <pyflowchart-code-diagram> BEGIN - auto-inserted, do not edit
82# Generated at (UTC): 2026-07-18T01:03:40Z
83# Flowchart(s) generated from this file:
84# * ``decide_verdict`` -> ``diagrams/code_diagrams/gco_mcp/mission/decide.decide_verdict.html``
85# (PNG: ``diagrams/code_diagrams/gco_mcp/mission/decide.decide_verdict.png``)
86# Regenerate with ``python diagrams/code_diagrams/generate.py``.
87# <pyflowchart-code-diagram> END
90__all__ = [
91 "build_revision_rationale_template",
92 "decide_verdict",
93]
96def decide_verdict(
97 session: SessionState,
98 iteration: IterationRecord,
99 now: datetime,
100) -> tuple[VerdictLabel, VerdictReason]:
101 """Return the deterministic Verdict for the in-progress iteration.
103 The cascade order is fixed (see module docstring). The first matching
104 branch wins: a session that has both run out of iterations and has
105 every Criterion met returns ``("terminate", "max_iterations")``, not
106 ``("complete", "criteria_met")`` — budget caps are evaluated before
107 completion so the operator can tell the loop ended because it ran
108 out of budget rather than because the goal was reached on the
109 closing iteration.
110 Note: When the prior Execute_Phase ran a scripted Strategy and the
111 sandbox cap fired, ``_execute_script`` writes
112 ``iteration["sandbox_terminated_reason"]`` and the cascade returns
113 that reason verbatim before anything else is consulted. The
114 sandbox limit is a true budget cap — the script ran out of wall
115 clock during execution — so it routes to a ``terminate`` verdict
116 on the same path as the ``BudgetControls``-driven caps below.
117 """
118 # 0. Sandbox-cap propagation. ``_execute_script`` stashes the
119 # reason on the in-progress iteration when the sandbox runner
120 # raised :class:`SandboxTerminated`. Reading the sentinel here
121 # means the engine's Execute_Phase can complete cleanly (no phase
122 # failure) while still routing the verdict to the budget-cap path.
123 sandbox_reason = iteration.get("sandbox_terminated_reason")
124 if sandbox_reason is not None:
125 return ("terminate", sandbox_reason)
127 # 1a. max_iterations — the +1 captures "this in-progress iteration
128 # would be the Nth one to land", so a session with budget=N and N-1
129 # already-recorded iterations terminates on the Nth's Decide_Phase.
130 # ``-1`` is the explicit "uncapped" sentinel; the validator
131 # already enforced that any other non-positive value is rejected.
132 max_iter = session["budget"]["max_iterations"]
133 if max_iter != -1 and len(session["iterations"]) + 1 >= max_iter:
134 return ("terminate", "max_iterations")
135 # 1b. max_wall_clock — pure time arithmetic; missing started_at
136 # means the session has not yet recorded its first iteration's
137 # start, so no wall-clock can be measured.
138 if _wall_clock_exceeded(session, now):
139 return ("terminate", "max_wall_clock")
140 # 1c. no_progress — the counter is incremented by the engine only
141 # on evaluated iterations, so a session with all-skipped checkpoints
142 # cannot terminate for stagnation. When the session has sampling
143 # enabled, the heuristic gets priority (step 4 below) so the
144 # sampler can revise the strategy before the loop terminates.
145 # Without sampling, ``adjust`` is purely informational and
146 # ``no_progress`` terminates immediately.
147 if session["no_progress_counter"] >= session["stagnation_threshold"]:
148 if not session.get("use_sampling"):
149 return ("terminate", "no_progress")
150 # With sampling enabled, fall through to the heuristic check
151 # below. If the heuristic fires, the sampler gets one more
152 # chance. If it doesn't fire (e.g., the tool sequence changed
153 # after a prior revision), terminate for stagnation.
154 _stagnation_pending = True
155 else:
156 _stagnation_pending = False
158 # 2. Completion — every required Criterion met AND nothing inconclusive.
159 if _completion_satisfied(session, iteration):
160 return ("complete", "criteria_met")
162 # 3. Cadence-skip — bail before the heuristic fires so off-cadence
163 # iterations don't ever produce ``adjust``. The iteration_index
164 # passed to ``should_evaluate_now`` is the 0-indexed position of
165 # the in-progress iteration (which equals the count of already-
166 # persisted iterations).
167 if not should_evaluate_now(session, len(session["iterations"]), now):
168 return ("continue", "cadence_skip")
170 # 4. Strategy_Revision_Heuristic.
171 unproductive, _heuristic_reason = _strategy_unproductive(session, iteration)
172 if unproductive:
173 return ("adjust", "heuristic_unproductive")
175 # 4b. Deferred stagnation — the counter hit the threshold but the
176 # heuristic didn't fire (e.g., the tool sequence changed after a
177 # prior sampled revision). Terminate now.
178 if _stagnation_pending:
179 return ("terminate", "no_progress")
181 # 5. Default.
182 return ("continue", "in_progress")
185# ---------------------------------------------------------------------------
186# Budget helpers — pure
187# ---------------------------------------------------------------------------
190def _wall_clock_exceeded(session: SessionState, now: datetime) -> bool:
191 """True iff ``now - session["started_at"] >= max_wall_clock_seconds``.
193 Returns False when ``started_at`` is absent — a session that has
194 never been transitioned out of ``pending`` cannot have exceeded any
195 wall-clock budget. The engine writes ``started_at`` on the first
196 iteration entry, so this guard only matters for the synthetic
197 "decide called before run_iteration" path used in unit tests.
199 Returns False when ``max_wall_clock_seconds`` is the explicit
200 ``-1`` "uncapped" sentinel — the operator opted out of the wall-
201 clock cap and the cascade should fall through to the next branch
202 rather than terminate spuriously.
203 """
204 started_iso = session.get("started_at")
205 if not started_iso: 205 ↛ 206line 205 didn't jump to line 206 because the condition on line 205 was never true
206 return False
207 max_seconds = session["budget"]["max_wall_clock_seconds"]
208 if max_seconds == -1:
209 return False
210 started = datetime.fromisoformat(started_iso)
211 return now - started >= timedelta(seconds=max_seconds)
214# ---------------------------------------------------------------------------
215# Completion check
216# ---------------------------------------------------------------------------
219def _completion_satisfied(
220 session: SessionState,
221 iteration: IterationRecord,
222) -> bool:
223 """True iff every required Criterion is met and none are inconclusive.
225 A session completes when all Criteria with ``required=True`` have
226 status ``met`` AND no Criterion (required or not) has status
227 ``inconclusive``. The ``required`` flag lives on the Criterion
228 declaration in ``session["criteria"]``; the per-iteration status
229 lives on ``iteration["criteria_evaluation"]``. The two are joined
230 by ``criterion_id``.
232 A session with zero declared Criteria can never complete on its own
233 — there are no required Criteria for the cascade to satisfy. The
234 operator drives such a session to terminal via ``mission_complete``
235 or a budget cap. We mirror that semantic here by returning False
236 when the criteria list is empty.
237 """
238 if not session["criteria"]:
239 return False
240 required_by_id = {c["criterion_id"]: c.get("required", True) for c in session["criteria"]}
241 for result in iteration["criteria_evaluation"]:
242 status = result["status"]
243 if status == "inconclusive":
244 return False
245 if required_by_id.get(result["criterion_id"], True) and status != "met":
246 return False
247 return True
250# ---------------------------------------------------------------------------
251# Strategy_Revision_Heuristic
252# ---------------------------------------------------------------------------
255def _strategy_unproductive(
256 session: SessionState,
257 iteration: IterationRecord,
258) -> tuple[bool, str]:
259 """Pure heuristic for the Strategy_Revision check.
261 Two clauses, evaluated in declaration order. The first match wins
262 so the returned reason is deterministic when both clauses fire.
264 * **Clause (a)** — the same ``tool_calls[*].tool_name`` sequence
265 has been used for the last 3 iterations (counting the in-progress
266 one) AND ``no_progress_counter >= ceil(stagnation_threshold / 2)``.
267 Needs at least 2 prior iterations to evaluate (3 total when the
268 current iteration is included). A scripted strategy contributes
269 an empty sequence so two scripts with the same body register as
270 "same sequence" — that's intentional: the heuristic flags repeats,
271 and an empty-sequence repeat across three iterations is a repeat.
272 * **Clause (b)** — the in-progress Observation contains at least
273 one ``errors`` entry that did not appear in the immediately
274 prior Iteration's Observation. Needs at least 1 prior iteration
275 to evaluate. Without a prior to compare to, "new" is undefined
276 and we return False.
278 Returns ``(False, "")`` when neither clause fires. When clause (a)
279 fires, the reason is ``"tool_sequence_repeating"``; when clause (b)
280 fires, ``"new_observation_errors"``. The reason string is
281 informational only — the Verdict's ``verdict_reason`` is always
282 ``"heuristic_unproductive"`` regardless of which clause matched.
283 """
284 # Clause (a): no_progress threshold AND tool-sequence repeat.
285 threshold = session["stagnation_threshold"]
286 half = math.ceil(threshold / 2)
287 if session["no_progress_counter"] >= half:
288 # Need at least 2 prior + the current = 3 total iterations.
289 prior = session["iterations"]
290 if len(prior) >= 2:
291 recent_three = [prior[-2], prior[-1], iteration]
292 sequences = [_tool_name_sequence(it) for it in recent_three]
293 if sequences[0] == sequences[1] == sequences[2]: 293 ↛ 297line 293 didn't jump to line 297 because the condition on line 293 was always true
294 return (True, "tool_sequence_repeating")
296 # Clause (b): new errors in the latest Observation vs the prior one.
297 if session["iterations"]:
298 prior_observation = session["iterations"][-1].get("observation") or {}
299 prior_errors = list(prior_observation.get("errors") or [])
300 current_errors = list(iteration["observation"].get("errors") or [])
301 for err in current_errors:
302 if err not in prior_errors: 302 ↛ 301line 302 didn't jump to line 301 because the condition on line 302 was always true
303 return (True, "new_observation_errors")
305 return (False, "")
308def _tool_name_sequence(iteration: IterationRecord) -> tuple[str, ...]:
309 """Extract the ordered tuple of ``tool_name``s from an iteration's strategy.
311 Returns an empty tuple when the strategy is a script (no
312 ``tool_calls``) or when ``tool_calls`` is missing. Two scripted
313 strategies therefore both produce ``()`` and compare equal — clause
314 (a) treats that as "same sequence", which matches the operator's
315 intent of flagging mechanical repetition regardless of mode.
316 """
317 strategy = iteration.get("strategy") or {}
318 tool_calls = strategy.get("tool_calls") or []
319 return tuple(str(call.get("tool_name", "")) for call in tool_calls if isinstance(call, dict))
322# ---------------------------------------------------------------------------
323# Revision rationale template
324# ---------------------------------------------------------------------------
327def build_revision_rationale_template(
328 session: SessionState,
329 iteration: IterationRecord,
330) -> str:
331 """Build the deterministic ``revision_rationale`` text for an ``adjust`` verdict.
333 Used both as the rationale on sessions with ``use_sampling=false``
334 and as the fallback rationale when sampling is rejected on a
335 ``use_sampling=true`` session.
336 Pure: depends only on persisted Session/Iteration fields, never
337 calls into the sampler or any other non-deterministic component.
339 The rendered text names the iteration index (1-indexed for
340 operator-friendliness), the heuristic reason, the unmet Criterion
341 ids (so the rationale points at the goal that's still moving), and
342 a one-line summary of the in-progress strategy (tool-name sequence
343 or ``"scripted strategy"``). The format is intentionally short and
344 machine-parseable — operators can grep it; no LLM is involved.
345 """
346 # Resolve the iteration index — the in-progress iteration has not
347 # been appended to session["iterations"] yet, so its 0-indexed
348 # position equals len(iterations) and the 1-indexed position is +1.
349 iteration_index_one_based = len(session["iterations"]) + 1
351 # Match the heuristic again so the rationale text matches whichever
352 # clause actually fired. Both calls are pure and cheap.
353 _, heuristic_reason = _strategy_unproductive(session, iteration)
354 if not heuristic_reason:
355 # decide_verdict only emits ``adjust`` when the heuristic fires,
356 # but the caller may invoke this template independently (e.g.
357 # the sampling-fallback path on a non-heuristic adjust) — fall
358 # back to a generic reason so the template stays usable.
359 heuristic_reason = "strategy_review_requested"
361 unmet_ids = [
362 result["criterion_id"]
363 for result in iteration["criteria_evaluation"]
364 if result["status"] == "unmet"
365 ]
366 unmet_summary = ", ".join(unmet_ids) if unmet_ids else "none"
368 strategy = iteration.get("strategy") or {}
369 if "script" in strategy:
370 strategy_summary = "scripted strategy"
371 else:
372 names = _tool_name_sequence(iteration)
373 strategy_summary = ", ".join(names) if names else "no tool calls"
375 no_progress = session["no_progress_counter"]
376 threshold = session["stagnation_threshold"]
378 return (
379 f"Strategy revised on iteration {iteration_index_one_based}: "
380 f"{heuristic_reason}. Unmet criteria: {unmet_summary}. "
381 f"Last strategy: {strategy_summary}. "
382 f"No-progress counter: {no_progress}/{threshold}. "
383 f"Adjusting approach for next iteration."
384 )