Call Replay Debugger

Use the State Program Debugger to replay a retained call on one clock, confirm the exact defect point, and inspect the state evidence around it.

Open a Call

Start the RIFF web server and open:

http://127.0.0.1:8765/state-program-debugger.html

Choose a call from Recent recorded calls. Each option shows its flow, local start time, duration, and exact session ID; newest calls appear first. Selecting an option copies its ID into the adjacent field and opens it immediately. You can still paste a known ID into Or paste a session ID and select Open. The list refreshes every ten seconds while the tab is visible; use the refresh icon to check immediately after a call ends.

The dropdown lists only retained calls with a session_manifest.json, so every listed item can be opened by the debugger. The page loads the normalized artifact through the same read-only engine used by the CLI and API. If you need to identify the current call for an AI or command-line invocation, read the ID from the text field or the page heading after the call opens.

What Moves Together

The replay cursor synchronizes:

• retained call audio, when available;
• caller and agent transcript bubbles;
• state visits, including repeated visits such as menu#2;
• playback request/start/completion/interruption/failure events;
• slot writes, transition commits, and evidence findings.

Play follows the selected audio track on the call clock. Pause freezes all lanes. The scrubber seeks audio and projects the last recorded event at that time. Step and Reverse pause playback, move one canonical event, and seek the selected track to that event's mapped sample.

The cursor inspector prints both values:

event 41 · state_entered · call 00:37.420 · sample 00:37.420

That equality is a claim only when the track badge is green. On an unverified legacy track, the inspector says nominal media instead of sample.

Audio Tracks

Use the audio-track selector to separate capture perspectives:

Do not infer conversational timing by comparing raw-track durations. Preserve those tracks so generation, codec, queueing, and capture defects can be heard in isolation. Use Aligned replay mix for synchronized playback. Older calls without clock metadata remain amber because their alignment cannot be recovered honestly after the fact. When every retained track is unverified, the debugger selects no audio and disables Play. Choose a Raw track explicitly to audition one capture perspective; it is not a synchronized conversation and Step/Reverse show nominal media rather than a proven sample position.

Confirm a Defect

1. Play or scrub until the audible problem begins.
2. Select Mark issue.
3. Use Earlier or Later to adjust the candidate point by one second.
4. Replay the window and answer Yes when it is the defect.

The server appends a revisioned marker containing session ID, call time, event ID, state visit, reporter, status, and determinism hash to debug_issues.jsonl. The browser keeps only a local display cache. That gives a human or AI one durable point to inspect instead of a prose description such as "somewhere near the second menu."

The marker is also encoded into the page URL as session_id, time_ms, and event_id. An AI can return that URL; opening it loads the call and seeks to the same evidence point. Confirmed markers add confirmed=1.

Find Repeated Audio

Browser and Telnyx playback paths use the same lifecycle vocabulary:

audio_requested → audio_started → audio_completed
                                 ↘ audio_interrupted / audio_failed

For browser static audio, acknowledgements originate at the actual client player callbacks and are persisted by the server. For Telnyx static audio, they originate at the outbound codec/send boundary. Matching content fingerprints within the bounded duplicate window produce DUPLICATE_AGENT_PLAYBACK.

Select Replay duplicate in the transport or Replay audio in the finding to audition a bounded window containing both starts. Show evidence seeks to the first playback event. The red playback spans show the first and duplicate requests on the same state/audio timeline.

Compare Repeated State Visits

Select a repeated visit such as menu#2 or cap_existing_request__choose_action#2. The comparison band appears directly below the current state and compares it with the immediately previous visit to the same state. It reports these semantic dimensions separately:

• entry path and Entry variable changes;
• the caller input the resolver acted on;
• source-labelled caller transcript observations;
• slot writes and agent playback spans;
• recorded Exit variables and winning edge.

The comparison ignores event IDs, sequence numbers, playback IDs, and clock values when deciding semantic equality. Those fields naturally differ on every visit and would create a false divergence. Entry path is shown as repeat topology, not automatically treated as a defect. The returned first difference is the earliest provable changed variable, input, playback, or exit in call time. If an earlier required evidence lane is unavailable, the debugger reports only an earliest known difference and does not enable a synchronized jump. Select Jump to first difference to move the one shared replay cursor, audio position, transcript highlight, and state inspector to that recorded point.

Late turn telemetry keeps two locations when synchronous transitions outrun event emission: causal_visit_id names the state visit that handled the input, while observed_visit_id names the visit active when the event was finally recorded. The debugger must compare by causal ownership. In the retained property-management call, “Status, please” was observed during choose_action#2 but causally belongs to choose_action#1; the second visit's first new input is the later no-match utterance.

Read the State Path

Each item under State visits shows the human-readable state title, visit ordinal, exact fired guard, exit target, and recorded exit reason. The fired guard is never guessed from the target or reason: the debugger joins the transition_verdict.winning_edge_id to the retained legal-exit table. If that identifier or table is missing, the page says Winning guard unavailable.

The selected state's Exit decision table lists every legal edge and marks only that authoritative edge as Fired. This matters when two guards share a target, such as denied and stalled both routing to leave-message. The raw recorded verdict remains available below the table.

Use Slot history in the right inspector to trace a variable across every state visit. Each row keeps the typed Entry and Exit value, boundary writer, whether the value changed, and the count of retained slot_write events. An absent variable, a redacted value, and an unavailable capture are distinct; the UI never renders any of them as null.

AI Use

Codex and Claude use the structured engine rather than reading screenshots:

.venv/bin/python scripts/state_program_debugger.py build \
  data/sessions/<session-id> --out /tmp/call.json

.venv/bin/python scripts/state_program_debugger.py command /tmp/call.json \
  --json '{"action":"inspect_timeline"}'

Available recorded operations include visit listing, state/variable/exit inspection, timeline/playback inspection, seek, step, reverse, and continue-until. Each response carries the cursor and determinism hash.

For a compact, human-readable view of a retained session directory, both AIs can use the wrapper around those same commands:

.venv/bin/python scripts/dbg.py data/sessions/<session-id> visits
.venv/bin/python scripts/dbg.py data/sessions/<session-id> explain 8
.venv/bin/python scripts/dbg.py data/sessions/<session-id> slot _route

visits prints the path with exact edge IDs and exit reasons. explain lists all legal exits and marks only the edge named by winning_edge_id. slot calls inspect_slot_history; it preserves absent, redacted, and unavailable cells rather than treating public placeholders as executable values.

The wrapper's what-if command is currently an explicitly labelled candidate-current-flow edge probe. It is not historical replay and is not connected to the browser. Do not use it as evidence of what a recorded call did. Browser mutation remains disabled until the capsule-backed subprocess runner can pin an engine revision, a flow revision, typed fixtures, and a simulation-only result namespace.

To export one visit for Codex, Claude, or a future isolated runner:

.venv/bin/python scripts/state_program_debugger.py command /tmp/call.json \
  --json '{"action":"export_capsule","visit_id":"menu#1"}'

The browser exposes the identical action as Capsule after a state visit is selected. Browser, CLI, and session API exports carry the same capsule and cursor hashes.

The returned riff.state_test_capsule/v1 contains the exact normalized flow and selected state definition, Entry boundary, a typed inventory of every mutable SessionContext field, state-owned events, structured caller/resolver/control/ audio/model fixtures, caller/agent transcript rows, playback spans, legal exits, recorded outcome, tool-result fixtures, audio and source references, and immutable flow/engine/build/runtime identities. The capsule and its complete execution-fixture set have separate SHA-256 hashes. Export reads only the recorded flow_snapshot.json; an older or tampered call fails with FLOW_SNAPSHOT_UNAVAILABLE instead of reading today's YAML.

Audio references carry the manifest and observed content hashes. Missing, changed, or stray unmanifested WAV files are unavailable rather than recorded. Tool results are executable fixtures only when a stable call ID and tool name join a complete result plus result schema; previews remain diagnostic evidence. The capsule also identifies the debugger projector revision that created it. A visit is limited to 2,000 events and a capsule to 4 MB.

recorded_reproduction.availability is independent of capsule export. Public slot placeholders are not executable values. Missing or redacted Entry slots, session history, turn/progress counters, recent/visited-state context, engine SHA, dependency lock, tool results, or caller-specific prompt overlays appear as typed gaps. The capsule remains useful for inspection, but no client may call such a run an exact historical reproduction.

Behavior-affecting feature flags are recorded separately from the Python and package lock. Live wall clock and randomness are currently unavailable execution inputs, so they remain blockers until an isolated runner supplies frozen fixtures.

To compare two visits without moving the cursor:

.venv/bin/python scripts/state_program_debugger.py command /tmp/call.json \
  --json '{"action":"compare_visits","baseline_visit_id":"menu#1","candidate_visit_id":"menu#2"}'

To deterministically move to the candidate-side event returned as the first semantic difference:

.venv/bin/python scripts/state_program_debugger.py command /tmp/call.json \
  --json '{"action":"rewind_to_first_divergence","baseline_visit_id":"menu#1","candidate_visit_id":"menu#2"}'

Both commands use the same engine as the browser and HTTP API. Equal evidence returns NO_RECORDED_DIVERGENCE; missing required evidence returns DIVERGENCE_UNAVAILABLE. Neither condition guesses or moves the cursor.

To review pauses without asking a model to interpret the call:

.venv/bin/python scripts/state_program_debugger.py command /tmp/call.json \
  --json '{"action":"inspect_pause_gaps"}'

The report separates caller-response wait, back-to-back agent output, and caller-input-to-agent-output delay. It prefers the recorded resolver input over nearby Whisper duplicates and marks superseded inputs rather than charging a later response to the wrong turn. The report's own analysis_execution is always zero model calls and zero tokens. Per-call model-token usage is marked unavailable until the live transport persists it in the session bundle.

Delays of two seconds or more from a resolved caller input to first agent output are returned as tuning_candidates. They are replayable in the browser's Analyze pauses panel and can jump the shared audio, transcript, state, and event cursor to the caller input. A candidate proves the interval, not its cause: model-generation, synthesis, and transport-enqueue spans remain typed unavailable until retained by the live session.

Visual Reading Order

The debugger uses an overview-plus-detail layout. The full-call scrubber is the overview. Below it, a 30-second rolling window keeps the playhead stable while four explicitly labelled lanes move through call time:

1. Caller: recorded caller transcript intervals.
2. Agent: audio-derived agent transcript and retained playback intervals.
3. State: FSM visit ownership on the same clock.
4. Finding: marked defects and deterministic pause-audit intervals.

Lane labels carry the meaning; color is redundant emphasis rather than the only key. Cyan means caller speech, green means confirmed agent audio evidence, blue means FSM state, and amber means a diagnostic interval. A legend remains visible above the lanes.

All lane spans and the amber playhead use the lane track's exact pixel bounds; the visible tick labels are generated from that same 30-second call-time scale. A transcript line is highlighted only while the cursor is inside its recorded start/end interval. During silence, no earlier line is presented as current. Clicking a transcript bubble seeks the synchronized call cursor to its start.

Aligned Whisper rows are also checked against their source-channel PCM before they become timeline bars. The hydrator measures 20 ms RMS frames, rejects a row with no supported speech frames, and trims silence-spanning Whisper bounds to the strongest contiguous speech region. Each retained row carries an acoustic_support record with its threshold, active-frame count, and original Whisper bounds. clock_status: aligned alone is not sufficient evidence that a word was spoken at that time.

Only a track whose manifest clock is aligned may move the call cursor. A raw or unverified WAV remains available for audition, but its sample time does not move transcript, state, findings, or events and the UI explicitly reports raw track unaligned.

The palette is semantic across both views: caller evidence is cyan, agent speech/audio is green, FSM state and state-owned events are blue, and the active cursor or diagnostic evidence is amber. Current highlighting adds an amber outline without replacing the underlying caller/agent/state color.

The state intersecting the gold playhead is deliberately more prominent than the surrounding evidence. Both Current State Program and the right-side state inspector use a four-pixel gold frame and clear gold halo while their interior remains state blue. The PLAYHEAD STATE edge label makes the relationship explicit without relying on color alone.

Only observations with an audio-derived start, end, and source channel become Caller or Agent lane duration bars. Gemini resolver observations remain visible in the conversation as point-in-time observations but are not drawn as if their event-emission timestamp were the caller's speech time. If the full-channel Whisper pass leaves an acoustically active caller region uncovered, the hydrator retranscribes that bounded region and validates the recovered segment against the source PCM before adding it.

The right panel shows the selected state as a small program: predecessor, successor, Entry variables, and values changed before Exit. Follow state is enabled during normal replay. Selecting a state visit disables it and pins the state and variables while audio can be inspected; enabling it again catches the panel up to the playhead. Detailed Entry/Exit tables remain below the transcript for inspection on demand. On a new state, its name changes immediately from the local recorded timeline while boundary variables load. Completed immutable state snapshots are cached in the browser, so revisiting a state does not rebuild and visually replace the panel again.

Recorded Versus Simulated

Recorded replay and state-capsule export are implemented and immutable. The purple Fork here control remains disabled until the isolated state runner is implemented and passes its no-network/no-write gates. A future fork will restore from the capsule's Entry boundary, apply one slot/input/tool/audio overlay, run one state to its exit, and compare the candidate with the recorded visit. It will not edit the historical audio or claim regenerated audio was heard by the caller.

Capture now distinguishes six identities that must never be collapsed:

1. normalized flow-content revision;
2. exact clean engine Git revision;
3. deployment build ID;
4. Python/dependency runtime lock;
5. behavior flags plus clock/randomness execution policy;
6. resolved call configuration such as transport, model, voice, STT, and codec.

A dirty runtime records the engine revision as unavailable. A phone call whose system prompt included caller-specific prior-call context freezes the authored flow first and records the dynamic overlay as redacted; exact LLM prompt replay is therefore unavailable even though deterministic FSM evidence remains useful.

The local debugger API validates loopback peer, Host, and browser Origin to prevent DNS-rebinding access to retained evidence. Non-loopback clients require RIFF_DEBUG_API_TOKEN; request bodies are capped at 256 KiB.

Evidence Gaps

The debugger reports unavailable evidence rather than filling it in. Older browser calls may have no event ledger or agent audio. New playback lifecycle capture applies only to calls recorded after deployment and server restart. AUDIO_CLOCK_UNAVAILABLE means WAVs exist but none can be placed on the event clock; the debugger must not auto-play their composite. Carrier-side Telnyx recording remains a separately authorized implementation packet; local lifecycle and wire evidence do not by themselves prove what the carrier delivered to the handset.

Debug the Debugger

The replay clock has three named coordinates:

1. Call time: milliseconds since the retained session start.
2. Event time: the canonical FSM/LLM/tool event's position on that call clock.
3. Media time: the selected WAV sample offset, mapped by sample_zero_call_ms in riff.audio_timeline/v1 metadata.

An aligned track must satisfy media_ms = call_ms - sample_zero_call_ms. The UI shows the mapping at the cursor and refuses the green aligned label when the manifest does not retain it.

The shift-left calibration test synthesizes a caller tone at 100 ms and an agent tone at 500 ms. It steps to the first event, steps forward to the second, then reverses. Each cursor position must land on the expected channel sample. This detects swapped channels, sign errors, hidden offsets, and forward/reverse drift without relying on a human listening judgment.

For a real-call validation, use this sequence:

1. Open the newest call and select Aligned replay mix.
2. Confirm the badge is green and the replay duration matches the call duration.
3. Play through one caller/agent exchange while watching transcript and state.
4. Pause on a known word, Step once, then Reverse once.
5. Confirm the call time, sample time, event, transcript, and audible window all return to the same evidence point.
6. Mark and share any mismatch as a debugger defect; do not diagnose the voice flow from a replay whose clock is amber.

Supporting Design

• State Program Debugger evidence contract
• Unified call capture and replay plan
• Browser playback debug and replay plan
• Implementation work packets