← Choice Health 2026-07-10-state-program-debugger-evidence-design.md raw .md

State Program Debugger Evidence Design

Date: 2026-07-10
Revision: 2, after Claude Fable adversarial review
Status: design contract ready for re-review; prototype and hydrator are not yet conformant

Decision

Build the debugger around a strict state-visit frame, not a loose time window. Each frame is a replayable state program with an immutable recorded entry snapshot, an ordered event stream, an immutable recorded exit snapshot, and a declared set of exit edges. A separate conversation lane preserves what the caller and agent said in chronological order.

The debugger must never infer a fact merely because it would make the display look complete. Every claim carries two independent labels:

This revision closes the four contract gaps found in the first adversarial review. It does not declare the current hardcoded prototype production-ready.

Product Outcome

An operator should be able to answer these questions without reading source code or correlating logs by hand:

  1. Which state visit owned this behavior?
  2. What values entered that visit, and where did they come from?
  3. What did the state read, write, clear, say, hear, and decide?
  4. What values existed at the selected program counter?
  5. What values left the state?
  6. Which exits were legal, which guards ran, which edge won, and why?
  7. What did the caller actually hear and say at that point in the call?
  8. Where did intended and observed behavior first diverge?
  9. Can the run be replayed or forked without confusing simulation with fact?

The first screen shows the caller story, current state breadcrumb, first divergence, evidence completeness, failure owner, and next action. Internal fields remain available one level below that summary.

State Visit Is the Unit of Debugging

A state ID is not a frame because the same state can be visited more than once. The unit is a state visit with a stable visit_id, derived as <state_id>#<ordinal> for old artifacts and emitted directly for new ones.

incoming edge
-> entry boundary and entry variables
-> state goal
-> ordered micro-events
-> exit edge evaluation
-> boundary writes and clears
-> frozen exit variables
-> transition commit
-> next state's entry boundary

The debugger cursor is (visit_index, micro_event_index). The display is a pure projection of the recorded prefix plus an optional simulation overlay.

Frame Ownership Contract

This contract resolves the review's R1 frame-attribution bug.

  1. A frame begins at its state_entered event. The initial frame begins at the session_started initial-state boundary plus the seeded-slot baseline.
  2. A frame ends at its matching state_exited event. Re-entry creates a new visit even when the state ID is unchanged.
  3. An event with an explicit visit_id belongs only to that visit.
  4. For legacy events without visit_id, the hydrator may use sequence position between matching entry and exit boundaries only when it does not conflict with an explicit state_id or trace node.
  5. transition_committed belongs to the outgoing boundary identified by its from_state and to_state, even when the current emitter records it after the target state_entered. It does not become a target-state event.
  6. A legacy completion event emitted after commit may attach to the source visit only when its from_state and turn_index match exactly one transition_committed event and visit. The attribution is labeled derived and cites both events. This covers current late locked_choice_evaluated rows without assigning them to the next state.
  7. Other events after state_exited.seq never belong to the prior frame.
  8. Outgoing edge writes and clears are shown in a boundary band. They are mutations caused by the source edge, appear in the source frame's exit snapshot, and also appear in the target frame's entry snapshot. They are not duplicated as two writes.
  9. Events that cannot be attributed without conflict go into an unattributed lane with an instrumentation finding. They are never forced into the nearest frame.
  10. A static announce state has caller input only when a recorded barge-in, DTMF, or caller-audio event occurred before that visit exited.

Property Menu Boundary Example

The prototype previously put the caller's "Status" utterance and _route=existing_request decision inside menu_full_announce. That is incorrect.

Visit Owns Must not own
menu_full_announce#1 intended menu text, generated/sent/heard menu audio, audio_done or audio_failed, edge writes such as _menu_announced=yes, commit to menu later caller utterance and menu resolver result
menu#1 entry value _menu_announced=yes, Gemini/Whisper/DTMF input, locked-choice resolver, _route mutation, guard verdict, commit to the selected capability prior announce playback events

Cross-frame leakage is a hydration error and a release-blocking debugger test failure.

Deterministic Stepping

The debugger has two cursors: an event cursor and an audio playhead. They never silently move each other.

The exit_boundary marker is explicitly derived. Its derived_from list contains the final transition verdict, boundary slot_write events, and the matching state_exited or transition_committed event. It is not represented as a recorded runtime event when one does not exist.

Variable Boundary Contract

The main variable table is:

Variable Entry At cursor Recorded exit Access evidence Last writer

Entry and Recorded exit are immutable recorded projections. A simulation edit may change only At cursor and later simulated results. The UI must render all declared and observed variables, including unset variables. A cell has an availability state of available, redacted, or unavailable; blank never means false, null, or absent.

Hydration Sources

This contract resolves R2 by naming the source and join for each headline field.

Debugger field Source Join/reconstruction rule
Visit boundaries session_started, state_entered, state_exited, transition_committed order by durable seq; pair by visit_id, or by state ID plus visit ordinal for legacy calls
Initial variables new session_slots_seeded baseline one row after all transport, identity, prior-call, simulator, and default seeds are applied, before the initial state executes
Entry variables baseline plus slot_write fold all writes before state_entered.seq; for the initial visit, use the seeded baseline as its logical entry boundary
At-cursor variables entry snapshot plus slot_write fold only writes owned by the visit whose seq is at or before the cursor, then apply the simulation overlay
Recorded exit variables entry snapshot plus slot_write fold all recorded visit and outgoing-boundary writes through the matching exit boundary; freeze the result
Mutation writer slot_write.writer, slot_write.kind show exact writer and mutation kind; never infer identify or a tool name from the value
Declared exits compiled flow metadata include transition declaration index, stable edge ID, to, when, reason, sets, and clears
Guard result guard_evaluated and transition_verdict join to visit and edge ID; preserve every evaluation, declaration order, result, and shadowed status
Winning exit final transition_verdict plus state_exited/transition_committed winner must agree with committed target; disagreement is an instrumentation defect
Audio and tools section-6 trace plus session event ledger join by visit_id; legacy fallback uses node/state and sequence interval with conflict checks
Caller evidence Gemini/export transcript, timed Whisper, DTMF/control trace join by audio range and visit; show selected source separately from observations

Baseline Coverage Requirement

The current phone path applies identity and prior-call values using self._session_ctx.slots.update(initial_slots) after session_started in riff/phone/telnyx_transport.py. Those values do not produce slot_write, so the current ledger cannot honestly reconstruct the initial entry snapshot.

Before the variable view is considered complete, call setup must pass all seed values through one instrumented seeding function and emit session_slots_seeded. Each baseline variable records its source writer, such as transport, caller_identity, replicant_identify, prior_call, simulator, or flow_default.

slot_filled is not a substitute. It is emitted later, covers public filled slots rather than every mutation, and describes collection evidence rather than the original writer.

The mutation ledger must also cover dictionary-valued changes and non-tool direct mutations. A build gate compares a boundary snapshot fingerprint with the folded baseline-plus-write result. A mismatch marks the variable view incomplete and reports the missing writer; it must not display guessed values.

Privacy and Value Availability

The complete variable inventory is required; unrestricted raw PII is not. Each value carries classification and availability. Authorized local artifacts may expose the recorded value under the existing session-retention policy. Public or shared artifacts use the PII-safe representation and mark the cell redacted. A redacted value cannot be edited in a simulation fork until an authorized source provides it.

Exit Edge Contract

The hydrator must extend _load_flow_metadata to emit every transition:

{
  "edge_id": "menu:1",
  "declaration_index": 1,
  "from": "menu",
  "to": "cap_existing_request__list_tickets",
  "when": "slot_equals:_route:existing_request",
  "reason": "caller chose existing_request",
  "sets": [],
  "clears": ["_required_tool_fired:cap_existing_request__list_tickets:replicant_list_tickets"]
}

The edge table at the cursor shows all declared edges and the latest verdict at or before the cursor. On exit it shows the final verdict, one winner at most, all evaluated losers, and all post-winner candidates as shadowed rather than false. Duplicate target states remain distinct through edge_id; target name alone is not a valid join key. The current 24-candidate truncation must either be removed for debugger artifacts or produce explicit incomplete evidence.

The committed target and final verdict must agree. No winner, multiple winners, an unknown edge, or a target mismatch is a release-blocking trace invariant.

Access Evidence Contract

This contract resolves R3 without claiming evidence RIFF does not have.

RIFF already emits guard-level runtime reads in guard_evaluated.slots_read from _SlotReadTracker. The reviewer statement that no read telemetry exists was therefore too broad. Coverage is limited to guard evaluation; prompt rendering, resolver logic, tools, and arbitrary state code do not yet have general runtime read instrumentation.

The access column may show only these evidence-backed labels:

The UI must not say ignored, because absence of telemetry does not prove a state ignored a value. Static references and runtime reads use different icons, labels, and provenance links. A future slot-access abstraction may add broader runtime slot_read events, but it is not required to hydrate an honest v1.

Canonical Micro-Event Vocabulary

This contract resolves R4. Only canonical events participate in Step Into and Reverse Step. Every event uses this envelope:

{
  "event_id": "bus:1842",
  "visit_id": "menu#1",
  "seq": 1842,
  "clock_ms": 12640,
  "kind": "locked_choice_evaluated",
  "provenance": "recorded",
  "source_refs": [{"artifact": "bus_events.jsonl", "seq": 1842}],
  "derived_from": [],
  "payload": {}
}

clock_ms is nullable. If the source has no trustworthy timestamp, the UI shows sequence order only. It never fabricates a precise-looking time.

Recorded Events

Recorded events are direct durable-artifact rows, normalized without changing their meaning:

The normalizer keeps source event names in payload.source_kind when multiple event dialects map to one debugger kind.

Derived Events

Derived events are deterministic projections with derived_from references:

The old prototype's audio_progress: 52% is not a micro-event. Progress is a WAV playhead computed from an actual audio range. When the WAV or range is missing, progress is unavailable.

Simulated Events

Simulation events are created only after a fork:

They use a separate sequence namespace, retain the recorded parent cursor, and can never satisfy a recorded evidence requirement.

Stable Ordering

Canonical order is (recorded seq, source suborder, event_id). Derived events anchor before or after a named recorded event and declare that placement. Simulation events order after their fork root. Rehydrating the same artifact must produce byte-for-byte identical event IDs and order.

Six-Beat Evidence Spine

The six beats summarize the detailed event stream without replacing it.

Beat Question Required evidence
Intended What should this visit do or say? versioned flow state, prompt/audio policy, legal exits
Generated What did the model, template renderer, or static asset produce? generation chunks, resolved text, asset identity, suppression evidence
Sent What reached the transport? playback ID, outbound trace, audio_pushed, completion/clear receipt
Heard What is proven audible? recorded WAV range, agent energy, silence, overlap, barge-in
Caller said What inputs were observed? Gemini, timed Whisper, DTMF/control, audio range
Decided What did RIFF act on? selected source, resolver, guard reads/results, slot writes, tools, verdict and commit

Each beat includes evidence status, provenance class, a short explanation, and source references. Sent may be unavailable while Heard is proven from the recording; the UI must preserve that distinction.

Conversation and Caller-Input View

The caller conversation is an always-visible downward lane, not a collection of bubbles scattered through technical rows. Agent speech, caller speech, DTMF, barge-in, and system decisions align vertically with the owning state frame.

Timed Whisper anchors caller audio. Gemini shows what the live model observed. DTMF and controls share the call-clock row. The resolver shows the text or token RIFF selected, its source, and the decision reason. Agreement collapses. Disagreement expands to show word differences, WER when supported, and the acted-on source. No selected source means Decided is unavailable, not empty.

Visual Information Hierarchy

Correct visits collapse by default. The selected visit and first divergent visit open automatically. Recorded is neutral, unavailable is amber, disproven is red, and simulated is purple with a persistent SIMULATION BRANCH header. Color is reinforced with text and icons.

Replay and Simulation Safety

Variable edits create an overlay at the current cursor. They never mutate the recorded entry or exit snapshots. Reversing before the fork root removes the overlay and restores the recorded trace. Recorded tool reads replay immutable fixtures; writes and external side effects remain disabled. A simulation can be exported only as a simulation artifact linked to its recorded parent.

Audio playback in production review uses the recorded WAV range. Browser speech is preview-only, is labeled simulated, and cannot prove Generated, Sent, or Heard.

Artifact Contract

state_audio_timeline.json remains the post-call summary source. The debugger adds a normalized state_program_debugger.json projection rather than making the browser join raw logs differently on every load.

Minimum top-level fields:

{
  "schema": "riff.state_program_debugger/v1",
  "session_id": "v3:example",
  "flow_id": "property_management",
  "recorded_engine_revision": "git:0123456789abcdef",
  "recorded_build_id": "riff-m3-20260710.4",
  "flow_version": "sha256:...",
  "completeness": "complete",
  "findings": [],
  "conversation": [],
  "visits": [],
  "unattributed_events": [],
  "sources": []
}

Each visit contains entry_edge, entry_snapshot, goal, micro_events, cursor_projection, evidence_beats, legal_exits, final_verdict, exit_snapshot, and exit_commit. Missing required sources downgrade completeness; they do not disappear.

Programmatic AI Debugging Contract

The visual debugger is one client of a deterministic debugger engine. Codex, Claude, and other authorized agents must use the same projection and simulation APIs as the browser; they must not scrape the DOM or reimplement state rules in their prompts.

Three Programmatic Modes

  1. Recorded navigation: open a retained artifact, list visits, seek, step, reverse, inspect variables/edges/evidence, and fetch recorded audio ranges. This mode executes no flow logic and can never change the trace.
  2. Isolated state simulation: create a fork at an entry boundary or cursor, change available variables or input events, and run an explicitly selected engine revision and flow revision. Tool reads use recorded or authored fixtures; external writes are disabled.
  3. Pattern runner: execute a bounded matrix of caller input, STT disagreement, DTMF, timeout, audio completion/failure, and tool-result fixtures against one state visit, then compare exits, variable deltas, audio output, and findings.

Recorded navigation answers "what happened." Isolated simulation and pattern runs answer "what would happen if this input or value changed." The result must always identify its mode.

State-Under-Test Capsule

Implementation status (2026-07-11): immutable call-time flow/runtime/call identity capture and export_capsule are implemented through the shared browser/API/CLI engine. Recorded navigation now includes authoritative winning_edge_id/exit-reason summaries and provenance-preserving inspect_slot_history; the browser and compact AI CLI consume those same commands. Candidate execution and historical reproduction remain disabled pending the isolated runner and safety gates; capsule availability is not an execution claim.

Any visit can be exported as a self-contained state-under-test capsule:

{
  "schema": "riff.state_test_capsule/v1",
  "flow_id": "property_management",
  "recorded_engine_revision": "git:0123456789abcdef",
  "recorded_build_id": "riff-m3-20260710.4",
  "flow_version": "sha256:...",
  "flow_definition": {},
  "visit_id": "menu#1",
  "state_definition": {},
  "entry_snapshot": {},
  "session_context": {},
  "recorded_events": [],
  "recorded_input_fixtures": {},
  "legal_exits": [],
  "tool_fixtures": [],
  "audio_refs": [],
  "evidence_refs": []
}

The capsule includes every recorded input dependency required to execute the state or marks it unavailable. It never silently reads today's flow when the recorded call used another flow version. The capsule does not claim that a flow definition captures Python behavior: historical reproduction also requires the recorded engine revision or a content-addressed engine artifact.

Execution Targets

Every command that executes state logic must select both revisions explicitly:

engine_revision and flow_revision have no execution defaults. Labels such as recorded, candidate, or HEAD are resolved to immutable hashes before the run begins. The response includes those hashes, runtime/dependency lock identity, fixture hash, compatibility/migration notes, and a determinism hash.

Machine Interface

The debugger engine exposes versioned structured commands through a local CLI and an authenticated local API. The browser is built on that interface.

Required actions are:

Each request supplies the session or capsule, cursor, action, arguments, and expected determinism hash. A command that executes logic also supplies an execution target with immutable engine and flow revisions. Each response returns the new cursor, mode, resolved execution target, projection, variable delta, exit decision, findings, evidence references, and new determinism hash. A stale expected hash fails rather than applying an edit to a different cursor.

Illustrative request:

{
  "session_id": "v3:example",
  "cursor": {"visit_id": "menu#1", "event_id": "bus:1842"},
  "action": "fork",
  "args": {
    "execution_target": {
      "purpose": "candidate_regression",
      "engine_revision": "git:fedcba9876543210",
      "flow_revision": "sha256:candidate-flow"
    },
    "overlays": {"_route": null},
    "caller_input": {"gemini": "status", "whisper": "status"}
  },
  "expected_hash": "sha256:recorded-prefix"
}

State Pattern Testing

An AI can generate a JSONL pattern file and ask the engine to run it against a single state capsule. A menu-state suite should cover at least:

The runner returns one row per pattern with the resolved engine and flow revisions, exact exit edge, entry-to-exit variable delta, generated/simulated audio reference, invariant failures, and a comparison against the recorded branch. An AI can therefore search for the smallest input change that reproduces or removes a defect, then promote that pattern into a deterministic regression test.

LLM and Audio Limits

Recorded model output and audio are replayable evidence. New model inference or newly synthesized audio is a simulation. A reproducible run pins model, configuration, prompt, tool fixtures, and captured output. An unpinned live model run is labeled simulated/non-deterministic and cannot pass a regression gate until its result is captured as a fixture or replaced by a deterministic assertion.

Agents may fetch and analyze a recorded WAV range, stop it, seek it, or compare it with a simulated render. A simulated render receives a new asset ID and never overwrites or proves the recorded Generated, Sent, or Heard beats.

AI Safety and Resource Limits

Programmatic sessions are read-only by default, local/authenticated, redaction aware, and bounded by maximum branches, events, wall time, and model budget. External tool writes, calls, texts, ticket mutations, and OTA actions require a separate explicit human authorization and are outside state simulation.

The packet-by-packet execution contract, target files, schemas, tests, model routing, and promotion states are in docs/design/2026-07-10-state-program-debugger-implementation-plan.md.

Implementation Sequence

  1. Frame correctness: assign visit IDs, split the announce and menu frames, quarantine conflicting events, and add the dedicated conversation lane.
  2. Metadata join: extend _load_flow_metadata with complete transitions, stable edge IDs, sets, clears, reasons, and static references.
  3. Variable truth: emit session_slots_seeded, cover all mutation shapes, fold baseline plus writes, and add the boundary fingerprint invariant.
  4. Canonical events: normalize bus, section-6 trace, transcript, audio, and derived events into the artifact contract.
  5. Debugger controls: drive step, reverse, breakpoints, audio, and forks exclusively from the normalized artifact.
  6. Programmatic engine: expose the same projection, state capsule, fork, version-resolved execution, pattern-runner, and comparison operations to the UI, CLI, and local API.
  7. Failure fixture: replace the happy-path center of the prototype with a retained failing call that exercises transcript disagreement or audio after state exit.
  8. Promotion: run focused unit/UI/API tests, replay fixtures, post-run audio gates, and a second adversarial review before production integration.

Acceptance Gates

  1. No event after a visit's exit can carry that visit ID.
  2. The property-menu fixture attributes "Status" and _route to menu, never menu_full_announce.
  3. Rehydrating the same artifact yields identical visit IDs, micro-event IDs, ordering, entry snapshots, and exit snapshots.
  4. An overlay edit changes only At Cursor and simulated descendants; recorded Entry and Recorded Exit remain byte-for-byte unchanged.
  5. Reverse before the fork root removes every simulated event.
  6. Every declared edge appears, including duplicate-target edges; exactly one edge may win.
  7. The final verdict, boundary writes, state_exited, and transition_committed agree.
  8. Guard reads cite guard_evaluated.slots_read; non-guard reads never claim runtime coverage.
  9. Missing baseline, truncated candidates, conflicting attribution, or fold mismatch produces incomplete, not a guessed value.
  10. No precise timestamp, audio percentage, transcript source, or variable writer exists without a source reference.
  11. Recorded, derived, unavailable, redacted, and simulated states remain distinguishable without relying on color alone.
  12. Desktop and mobile screenshots show no overlap; all controls remain usable with the longest state and variable names in the property flow.
  13. UI, CLI, and API commands produce the same projection and determinism hash for the same artifact and cursor.
  14. A state capsule runs without reading mutable production flow files or making an external write.
  15. A pattern matrix can reproduce a recorded branch, isolate a failing counterexample, and export that counterexample as a deterministic test.
  16. Recorded reproduction fails as unavailable when the historical engine or flow revision cannot be resolved; it never falls back to current code.
  17. Recorded and candidate runs report independent immutable engine, flow, runtime, and fixture identities so an AI can isolate which layer changed.

Review Resolution Matrix

Review item Resolution in this contract Implementation consequence
R1: next-state evidence leaked into announce frame strict visit boundaries, conflict quarantine, and corrected property-menu example split current prototype frames and add cross-frame semantic test
R2: boundary variables and exits lacked hydration source explicit baseline/write fold, flow-transition metadata, verdict/commit joins, and identity baseline finding add seeded baseline, extend metadata, hydrate both headline tables
R3: access claims were unbacked use existing guard slots_read, static-reference labels, and remove ignored change access column labels; broader runtime reads remain unavailable
R4: micro-events were unspecified canonical recorded/derived/simulated vocabulary, envelope, ordering, and source rules replace synthesized rows and fabricated audio progress

Recommended items are also incorporated: a failing-call fixture, an always-visible conversation lane, explicit owner/next action, and semantic UI tests.

Current Prototype Status

web/state-program-debugger-mockup.html demonstrates the interaction direction but remains nonconformant until the implementation sequence above is complete. In particular, hardcoded values, browser speech, and the announce/menu frame leak must not be mistaken for hydrated debugger evidence.

The implementation may proceed only through the shift-left steps above. A production integration decision requires the focused gates and an explicit second-review GO.