Debugger Audio Causality and Explain-Moment Design

Date: 2026-07-12
Status: Approved for implementation by Claude CL2C-002
Build status: Not implemented
Extends: docs/design/2026-07-10-state-program-debugger-evidence-design.md
Audio truth contract: docs/design/2026-07-10-unified-call-capture-replay-plan.md
Motivating call: v3:FXI4R9RYo0Y18AZQHu4uMjHGgcuiQRzHoPLHWGvx4Il0sp1cUa4iTg

Decision

Add a transport-neutral output-attempt ledger, join it to the existing cross-transport playback lifecycle, and add a bounded, deterministic explain_moment projection to the existing State Program Debugger. Do not create a second timeline. Static playback, generated Gemini audio, entry lines, nudges, and restart attempts all normalize into one caller-facing output-attempt model and remain joined to the existing canonical event cursor.

The design has three required properties:

1. It distinguishes an FSM requesting the same content twice from a transport replaying one attempt twice and from a model restarting after interruption.
2. It records enough non-slot state context to explain guards such as stalled without guessing internal counters.
3. It lets a human, Codex, or Claude request a compact causal explanation of one moment without loading a 100,000-token timeline.

Mutation remains disabled. The isolated capsule-backed simulation runner is a later packet and consumes this evidence; it is not part of this capture change.

Effectiveness Target

“200% better” means a threefold improvement against the current retained-call workflow, measured rather than inferred:

• one audible defect is localized with latest plus explain_moment in at most two debugger commands;
• the default evidence returned to an AI is at most 10 KB total, versus the roughly 148,000-token unfiltered timeline for the motivating call;
• a human can open a durable issue marker and reach the first causal divergence in at most three interactions without reading raw JSON;
• the compact explanation and full artifact name the same state, input, decision, and output-attempt IDs in deterministic tests.

These are product acceptance measures, not a claim that instrumentation alone fixes the caller-facing behavior.

Evidence From the Last Call

The retained call has an aligned replay WAV, caller and outbound-agent channels, 47 timed Whisper rows, 526 bus events, and static playback lifecycle evidence. That was enough to prove three distinct failure classes.

The live log confirms the second generated response again begins “Before we start:” and then continues the full disclaimer. The existing DUPLICATE_AGENT_PLAYBACK detector finds the two static retry visits but misses the generated restart. The existing pause audit also reasons primarily from static playback spans and misses dynamic prompt boundaries.

This proves the current evidence is useful but not yet self-explanatory.

Terminology

• Logical output: the caller-facing utterance RIFF intends to deliver, such as the consent disclaimer. One logical output can have multiple attempts.
• Model request: one send_client_content injection with a typed purpose and output expectation. Policy/context requests may forbid caller-facing output; a model response to one is observable but not retroactively “intended.”
• Output attempt: one contiguous provider generation effort for that logical output. An interruption followed by a provider restart creates a new output attempt.
• Playback attempt: one transport delivery effort, identified by the existing playback_id, for static audio or a generated output attempt. One output attempt may have more than one playback attempt only when an explicit retry, resume, or duplicate is recorded.
• Generated receipt: PCM/text received from the model before a transport accepts it. This is not sent or played evidence.
• Transport playback: static or generated audio acknowledged at the named browser-player or outbound phone boundary. It is sent/played evidence at that boundary, not carrier confirmation.
• Carrier recording: the retained call recording perspective. It is the strongest available evidence of what traversed the call, but it is still labelled separately from a handset-side acknowledgement.
• Causal parent: the event or request that caused a later event. Timestamp proximity alone is not causality.
• Observed visit: the visit active when telemetry was emitted.
• Causal visit: the visit whose input, decision, or output produced it.

One Canonical Output-Attempt Model

The normalized artifact adds output_attempts. Existing playback_spans remains a compatibility projection until clients migrate; it must be derived from output_attempts, never maintained independently.

{
  "schema": "riff.output_attempt/v1",
  "model_request_id": "req:v3-example:19",
  "request_kind": "locked_choice_prompt",
  "output_expectation": "required",
  "logical_output_id": "out:v3-example:14",
  "output_attempt_id": "attempt:v3-example:14:2",
  "playback_ids": ["play:v3-example:14:2:1"],
  "parent_attempt_id": "attempt:v3-example:14:1",
  "restart_reason": "caller_barge_in",
  "source_kind": "model_audio",
  "transport": "telnyx_outbound",
  "causal_visit_id": "cap_start_claim__consent#2",
  "observed_visit_id": "cap_start_claim__consent#2",
  "state_id": "cap_start_claim__consent",
  "decision_epoch": 14,
  "input_id": "input:v3-example:4",
  "selected_source": "whisper",
  "decision_reason": "whisper_primary",
  "model_response_id": "gemini-response:7",
  "requested_event_id": "bus:389",
  "started_event_id": "bus:395",
  "terminal_event_id": "bus:402",
  "requested_ms": 51527,
  "first_text_ms": 55420,
  "first_audio_ms": 55429,
  "last_audio_ms": 62920,
  "terminal_ms": 62925,
  "outcome": "completed",
  "content_fingerprint": "hmac-sha256:...",
  "normalized_prefix_fingerprint": "hmac-sha256:...",
  "audio_bytes": 401280,
  "audio_duration_ms": 8360,
  "attempt_confidence": "provider_signalled",
  "provenance": "recorded",
  "source_refs": ["bus:395", "bus:402"]
}

Identity Rules

1. logical_output_id identifies one intended caller-facing message across interruption, retry, resume, or regeneration.
2. output_attempt_id identifies one contiguous generation/playback effort.
3. parent_attempt_id is present only when a recorded restart, resume, or replacement relationship exists.
4. model_request_id identifies every session-side model injection and records request_kind plus output_expectation=required|allowed|forbidden|unknown.
5. model_response_id comes from the provider when available. Otherwise RIFF allocates a connection-local monotonic response ordinal and labels it derived_local.
6. decision_epoch is the epoch active when the logical output was requested.
7. input_id joins the selected caller input to its resolver, transition, and resulting output. It is absent for unsolicited greetings and timer actions.
8. causal_visit_id never changes because telemetry arrived late.
9. IDs are session-scoped, bounded, and contain no raw caller text.
10. playback_id identifies one transport delivery attempt and always names its output_attempt_id; static sources allocate a synthetic output attempt so the artifact shape remains uniform.

Using only turn_index is prohibited. The motivating call recorded two genuine audio starts with the same turn_index=4 because the per-turn scratch buffer reset while SessionContext.turn_count did not advance.

Identity allocation is evidence-driven:

Every session-side send first allocates a bounded model_request_id. The semantic identity key is (session_id, causal_visit_id, request_ordinal); its encoded ID may use stable ordinals/tokens so it always satisfies the existing 160-character safe-ID limit.

Content equality or timestamp proximity never causes two attempts to share a logical output. A provider ID is supporting identity evidence, not a substitute for the RIFF logical-request ID.

Raw Durable Event Family

AC-00 freezes these additive events; implementation agents must not choose a different vocabulary:

model_request_recorded.dispatch_outcome is accepted or failed. The wrapper allocates identity before the call, records accepted only after the awaited provider call returns, and records failed best-effort before re-raising the same exception. It does not swallow, translate, retry, or reorder the call.

The existing assistant_first_audio event remains a compatibility/timing row and gains causal IDs. It is not an attempt lifecycle. The existing audio_requested/started/completed/interrupted/failed family remains the only playback lifecycle.

Capture Contract

Use the existing canonical lifecycle names from riff/playback_evidence.py for every playback attempt:

audio_requested
  -> audio_started
  -> audio_completed | audio_interrupted | audio_failed

Generated audio extends these events with source_kind=model_audio; it does not invent a second playback vocabulary. Provider response boundaries remain provider evidence and are normalized into output_attempts. They must not be relabeled as playback evidence.

Evidence Stages

The same audio has different truth at different capture points. Every row names its stage; the UI and detectors may not collapse them:

LiveSession.record_audio_out currently runs before browser await ws.send(payload) and before Telnyx _emit_pcm24_bytes; therefore it proves model_received, not outbound_accepted, client_playback, or carrier_recorded. Instrumentation must preserve that distinction.

Required Payload

Every playback lifecycle event carries:

• logical_output_id, output_attempt_id, and playback_id;
• model_request_id, request_kind, and output_expectation when model-backed;
• playback parent/retry identity when applicable;
• source_kind and transport;
• evidence_stage and an acknowledgement source;
• state_id, causal_visit_id, and decision_epoch;
• input_id and model_response_id when applicable;
• session-scoped content and normalized-prefix fingerprints;
• an exact event timestamp and source provenance.

Terminal events also carry submitted or played bytes, sample rate, channel count, derived audio duration, text fingerprint, and a bounded reason. Exact output transcript continues to live in the transcript evidence lane; lifecycle rows do not copy a second full transcript.

Generated-Audio Capture Points

The implementation must use these choke points rather than scattered log parsing:

An AST/source gate must reject new direct send_client_content calls in riff/live/session.py outside the wrapper. This follows the existing rule that hand-copied live paths drift.

An independent AST/source gate must reject raw self.turn.reset() outside _reset_turn_scratch. The output-attempt tracker is owned by LiveSession, not TurnAccumulator, so a scratch reset cannot erase causal identity.

Attempt-Boundary Rule

For a logical output with an existing started output attempt, a new attempt may open only after one of these recorded boundaries:

1. an explicit provider interruption;
2. a changed provider response ID;
3. a typed RIFF retry/resume action; or
4. a _reset_turn_scratch(reason) boundary followed by fresh first text/PCM.

If the prior attempt lacks a provider terminal at boundary 2 or 4, record a derived interruption with attempt_confidence=derived and a typed reason such as derived_response_restart or derived_scratch_reset. Provider-signalled interruption uses attempt_confidence=provider_signalled. A time gap or matching text alone is never a response boundary. If no boundary can be established, keep the attempts separate and expose OUTPUT_ATTEMPT_LINK_UNAVAILABLE rather than inventing a parent link.

Lifecycle Invariants

1. One playback_id has at most one audio_started per evidence stage.
2. One started playback attempt has exactly one terminal event in a finalized call, or a typed PLAYBACK_TERMINAL_UNAVAILABLE finding.
3. One started output attempt has one provider terminal or typed OUTPUT_ATTEMPT_TERMINAL_UNAVAILABLE evidence before scratch identity is discarded.
4. Completion after interruption is invalid for the same stage and attempt.
5. A model restart uses a new output-attempt ID; a transport retry uses a new playback_id. Each links only to the matching parent kind.
6. State transition never silently reassigns an attempt's causal visit.
7. Static audio behavior and bytes remain unchanged by instrumentation.
8. Event emission failure never blocks caller audio, but it produces a capture completeness finding.

State-Context Evidence

Slots are not the entire FSM context. The debugger adds a recorded state_context_snapshot at state entry and exit with behavior-affecting fields:

{
  "type": "state_context_snapshot",
  "phase": "entry",
  "visit_id": "cap_start_claim__consent#1",
  "state_id": "cap_start_claim__consent",
  "turn_count": 3,
  "no_progress_streak": 2,
  "max_no_progress_loops": 2,
  "decision_epoch": 11,
  "recent_states": ["menu_retry", "menu"],
  "pending_transition": null,
  "active_logical_output_id": null,
  "active_output_attempt_id": null,
  "context_fingerprint": "sha256:..."
}

The snapshot also inventories unavailable fields. It is separate from state_slot_snapshot because Python attributes and slots have different read and write semantics.

guard_evaluated gains context_reads for non-slot values read by the guard. For stalled, that evidence must include no_progress_streak and max_no_progress_loops. slots_read=[] must no longer imply that the guard read no behavior-affecting data.

Causal-Link Contract

New calls carry explicit links:

input_id
  -> transcription observations
  -> selected observation + selected_source + decision_reason
  -> locked_choice_evaluated/tool decision
  -> slot writes
  -> transition_verdict/transition_committed
  -> model_request_id + request_kind + output_expectation
  -> logical_output_id
  -> output_attempt_id(s)
  -> playback_id(s) at named evidence stages
  -> carrier-recorded transcript/audio interval

Each link cites source events. A legacy artifact may derive a link only when the join is unique and non-conflicting. Derived links state the rule. Ambiguous links remain unavailable.

The compact input_id record contains selected_source and decision_reason because those scalars determine which observation acted on the FSM. It cites, rather than duplicates, the complete Gemini/Whisper transcript_sources block. This preserves the current whisper_primary and gemini_backup decisions.

Artifact normalization is a pure function of recorded events ordered by (seq, source_suborder). Builders may not depend on current YAML, wall clocks, random values, process hash order, or Python set iteration. All indexes used by explain_moment are deterministic artifact projections.

explain_moment Command

explain_moment is a read-only deterministic projection over the canonical artifact. It executes no flow logic and calls no model.

Request

{
  "action": "explain_moment",
  "anchor": {"event_id": "bus:395"},
  "window_before_ms": 4000,
  "window_after_ms": 10000,
  "detail": "summary",
  "expected_hash": "sha256:..."
}

Exactly one of event_id or time_ms is required. detail is summary, evidence, or raw_refs. The default response is capped at 4 KB canonical JSON; evidence detail is capped at 32 KB and paginated. Raw events are returned only by reference or an explicit filtered request.

Response

{
  "schema": "riff.explain_moment/v1",
  "mode": "recorded",
  "anchor": {"event_id": "bus:395", "clock_ms": 55429},
  "state": {
    "visit_id": "cap_start_claim__consent#2",
    "entry_ms": 51526,
    "exit_ms": 68275
  },
  "caller_input": {
    "input_id": "input:4",
    "selected_text": "<recorded transcript reference>",
    "selected_source": "whisper",
    "decision_reason": "whisper_primary",
    "source_refs": ["bus:400"]
  },
  "decision": {
    "winning_edge_id": null,
    "context_reads": []
  },
  "output": {
    "logical_output_id": "out:14",
    "attempts": ["attempt:14:1", "attempt:14:2"]
  },
  "first_divergence": {
    "code": "MODEL_OUTPUT_RESTART_REPEATED_PREFIX",
    "event_id": "bus:395",
    "confidence": "recorded_lifecycle_plus_text"
  },
  "evidence_gaps": [],
  "next_commands": [
    {"action": "inspect_output_attempt", "output_attempt_id": "attempt:14:2"}
  ],
  "analysis_execution": {"model_calls": 0, "input_tokens": 0, "output_tokens": 0}
}

The response reports facts and deterministic findings. It does not claim an LLM, network, or transport root cause when the required stage telemetry is missing.

Deterministic Findings

The debugger replaces the overly broad duplicate label with mechanism-aware findings.

For legacy calls, DUPLICATE_AGENT_PLAYBACK remains available with its older, weaker meaning. New calls use the mechanism-aware codes.

UI Contract

The existing synchronized timeline remains the overview. The Agent lane renders one band per playback attempt, grouped by logical output and output attempt. A thin labelled receipt tick may precede it when model receipt and transport acceptance differ materially:

• green: completed;
• amber: interrupted;
• red: failed or invalid lifecycle;
• purple: simulated, only after the isolated runner exists.

Linked output restarts share a small connector and logical-output label. Transport retries show distinct playback_id labels inside the same output attempt group. Color is redundant; every band includes status text, evidence stage, and an accessible title.

At the playhead, the right inspector shows one downward causal story:

Caller spoke
Selected STT source
Resolver matched
State/guard decision
Model request intent and output expectation
Output attempt 1 started
Caller interrupted
Output attempt 2 restarted
Playback submitted to transport
Browser playback or carrier evidence: available/unavailable
Repeated prefix detected

The selected state breadcrumb, Entry/Exit variables, and context values remain visible. The UI does not create cards inside cards and does not add a second independent cursor.

Selecting Explain this moment calls the shared command. Selecting Mark problem stores the anchor plus finding code and output-attempt IDs in the existing durable issue marker. An AI-created marker starts as proposed; the operator can replay its bounded interval and select Confirm this point. Confirmation changes marker status, not recorded evidence. The resulting URL restores the same cursor, interval, and explanation.

AI Contract

Codex and Claude use the same API as the browser. The minimum high-level command set is:

latest_call(flow_id)
explain_moment(event_id | time_ms)
trace_cause(event_id)
list_output_attempts(visit_id | time_range)
inspect_output_attempt(output_attempt_id)
inspect_context_history(field)
compare_visits(baseline_visit_id, candidate_visit_id)
mark_defect(anchor, finding_code)
export_capsule(visit_id)

Every list command is bounded and paginated. Responses default to compact summaries with source references. An AI should never need to load the entire timeline to answer where one audible failure began.

mark_defect created by an AI records status=proposed, the anchor, bounded replay interval, finding code, rationale, and source references. Only an explicit operator action may change it to confirmed; neither status changes the underlying artifact.

explain_moment resolves through prebuilt indexes keyed by event, visit, input_id, logical output, output attempt, and playback ID. The causal join is bounded by the requested clock window and referenced IDs; it must not scan or serialize the full timeline on each request.

Simulation Boundary

This design does not enable browser mutation. A future simulation may change a typed Entry value, caller input fixture, or output fixture only after exporting an immutable capsule and selecting exact engine/flow revisions. Simulated audio attempts use a separate namespace and purple presentation. They never append to the recorded ledger or call external tools.

Acceptance Gates

1. A generated response that is interrupted and restarted produces two attempt IDs, one parent link, and one terminal event per attempt.
2. Rehydrating the same ledger produces byte-identical output_attempts and determinism hash.
3. The static playback fixtures remain behaviorally and byte-for-byte unchanged.
4. The motivating consent sequence yields MODEL_OUTPUT_RESTART_REPEATED_PREFIX, not TRANSPORT_DUPLICATE_ATTEMPT.
5. The two menu retries yield FSM_REPEATED_OUTPUT_REQUEST and preserve both causal visits.
6. A consent state entered with no_progress_streak=2 records that value and reports STALE_PROGRESS_GUARD_ON_ENTRY when stalled fires before owned caller input.
7. A dynamic prompt followed by 60 seconds of silence appears in the pause audit.
8. explain_moment(detail=summary) is at most 4 KB and uses zero model calls.
9. Browser, CLI, and HTTP responses are byte-identical after canonical JSON encoding.
10. A stale expected hash, unknown event, ambiguous legacy link, or incomplete lifecycle fails closed with a typed error.
11. UI Play, Pause, Step, Reverse, transcript, state, and attempt selection use the one shared call clock.
12. A Playwright test proves an operator can jump from a repeated-prefix finding to the first attempt, play through the restart, and inspect its causal chain.
13. With RIFF_DEBUG_OUTPUT_ATTEMPTS=0, fake-client and retained-browser comparisons produce byte-identical audio and pre-existing bus event streams.
14. Source gates fail when a direct send_client_content or raw self.turn.reset() is introduced outside its approved wrapper.
15. Every scratch-reset path terminalizes the active attempt or records typed terminal-unavailable evidence before fields are cleared.
16. When the provider interruption bit is absent, a recorded response/reset boundary produces a derived restart with lower confidence; a time gap alone does not.
17. Whisper-primary and Gemini-backup fixtures retain the acted-on source and decision reason through input_id into explain_moment.
18. A model-received PCM event before a failed browser/Telnyx send never appears as outbound-accepted, client-played, or carrier-recorded evidence.
19. The normal path records logical output -> output attempt -> playback ID at its exact truth stage; no detector treats stage multiplicity as duplication.
20. A model response to output_expectation=forbidden remains linked to its request and produces UNEXPECTED_OUTPUT_FROM_NON_SPEAKING_REQUEST; it is not rewritten as an intended FSM prompt.

Rollout

1. Land schema and pure normalization tests.
2. Add request identity plus provider-attempt capture behind RIFF_DEBUG_OUTPUT_ATTEMPTS=1.
3. Add browser and phone playback-stage capture without changing audio bytes or send ordering.
4. Record local fake-client and browser calls; compare emitted audio bytes and pre-existing events with the flag off/on.
5. Make capture default after lifecycle invariants pass.
6. Add debugger projections and UI.
7. Run one Telnyx OTA validation only after explicit human GO.
8. Keep legacy projections until retained pre-schema calls age out.

Non-Goals

• Proving that a handset speaker rendered a packet without carrier/handset acknowledgement.
• Changing Gemini prompting, FSM transitions, retry policy, or barge-in behavior in the instrumentation packets.
• Calling an LLM inside explain_moment.
• Enabling external tool writes or production calls from a debugger fork.
• Treating generated audio as static playback or inferring content from duration.

CL2C-001 Resolutions

Open Review Questions

1. Does the Gemini SDK expose a stable response ID and explicit interrupted bit in every supported model/version, or must RIFF allocate the local response ordinal for some paths?
2. What is the smallest flag-protected browser control message that binds binary PCM frames to a playback_id without changing audio frame bytes or ordering?
3. Can the Telnyx send helper report first-success/final-success/failure without changing its existing exception behavior, or must telemetry wrap ws_send inside _emit_pcm24_bytes_locked?
4. Which non-slot SessionContext fields can current guards read beyond the initial required inventory?
5. Should STALE_PROGRESS_GUARD_ON_ENTRY remain debugger-only initially, then become a flow/compiler lint in a separately reviewed packet?