← Choice Health 2026-07-12-debugger-audio-causality-implementation-plan.md raw .md

Debugger Audio Causality Implementation Plan

Date: 2026-07-12
Status: Approved for implementation by Claude CL2C-002
Design: docs/design/2026-07-12-debugger-audio-causality-design.md
Coordination: docs/toclaude/2025.07.12 and docs/tocodex/2025.07.12

Goal

Make one audible failure explainable from one bounded debugger command and one synchronized UI cursor. A human, Codex, or Claude must be able to distinguish:

The first release is recorded observability only. It must not change prompts, FSM behavior, audio bytes, transport ordering, or production tool effects.

Rules for Implementation Agents

  1. Read the design, this plan, relevant fixlog entries, and both coordination mailboxes before editing.
  2. Work one packet at a time. Do not combine capture, hydration, UI, and caller behavior in one commit.
  3. Search the fix journal before each behavioral or telemetry fix and add an FX entry after the fix.
  4. Do not infer historical facts from current YAML or current Python.
  5. Do not add a second stored timeline. Normalize into the existing artifact.
  6. Never label record_audio_out as sent or played evidence; both current transports call it before their send boundary.
  7. Do not enable the purple Fork UI in these packets.
  8. No OTA call without explicit human GO; use fake clients, fixtures, browser simulation, and retained replay overnight.
  9. Never stage unrelated dirty files. At plan creation these include Claude's mailbox, packs/space_channel_widgets.pack.yaml, and static-audio assets.
  10. Each packet ends with focused tests, a narrow commit, and a mailbox result.
  11. If a packet discovers that caller-facing ordering must change, stop and split that into a separately reviewed behavior packet.

Ownership

Lane Owner Files
Request/provider capture Claude riff/live/live_client.py, riff/live/session.py, capture-focused tests
Browser/phone playback capture Claude riff/live/transport.py, riff/phone/telnyx_transport.py, web/live/*, web/voice-dispatcher.js, capture tests
Shared evidence schema Claude proposes, Codex reviews riff/playback_evidence.py, schema tests
FSM context capture Claude/service lane riff/state_manager.py, new or existing context-ledger helper, tests
Artifact normalization Codex riff/postcall/state_program_artifact.py, artifact tests
Debugger commands/detectors Codex riff/state_program_debugger.py, CLI/service/HTTP tests
Browser UI and operator docs Codex web/state-program-debugger.*, Playwright, Markdown/HTML
Caller-flow behavior fixes Claude, separate plan live/FSM/compiler files; never mixed with telemetry packets

Do not edit another owner's files without a mailbox handoff naming the exact file and reason.

Verified Starting Point

These facts were rechecked on the plan's working tree. Function names are the contract; line numbers may move during implementation.

Fact Current evidence Consequence
Caller-facing session sends 10 direct self.client.send_client_content(...) calls in riff/live/session.py AC-01 wraps these 10; the source gate is scoped to this file
Inbound browser text send riff/live/transport.py::_browser_to_session sends caller text directly to the provider This is input, not a logical caller-facing output, and must not receive an output ID
Scratch clears 11 direct self.turn.reset() calls in riff/live/session.py AC-01 replaces all 11 with the reasoned reset wrapper
Browser model PCM riff/live/transport.py::_session_to_browser calls record_audio_out before await ws.send The first call is model receipt; only the successful send boundary proves outbound acceptance
Phone model PCM TelnyxPhoneTransport._emit_audio_out calls record_audio_out before _emit_pcm24_bytes; actual sends occur in _emit_pcm24_bytes_locked Pre-codec receipt, outbound wire submission, and carrier recording remain separate
Browser generated player web/voice-dispatcher.js enqueues anonymous PCM; web/live/audio-player.js and pcm-up-processor.js report no per-attempt start/drain today AC-03A needs a bounded begin/end envelope and worklet acknowledgement
Guard reads riff/state_manager.py::evaluate_guard reports slot reads only; _guard_stalled reads no_progress_streak and max_no_progress_loops AC-04 adds explicit non-slot context_reads before making a stale-context claim

Packet Dependency Graph

AC-00 schema + synthetic fixture
  -> AC-01 request identity + structural gates
  -> AC-02 provider output-attempt capture
       -> AC-03A browser playback truth
       -> AC-03B phone outbound truth
       -> AC-04 non-slot state-context capture
  AC-03A + AC-03B + AC-04 -> AC-05 artifact normalization
  -> AC-06 mechanism-aware findings
  -> AC-07 explain_moment + compact AI CLI
  -> AC-08 browser visualization
  -> AC-09 regression promotion seam
  -> AC-10 local/OTA validation and rollout

AC-03A and AC-03B may run in parallel after AC-02. AC-04 may run in parallel with AC-03 only when ownership does not overlap. AC-05 begins only after the schema and capture examples are frozen.

AC-00: Freeze Schema and Synthetic Evidence

Owner: Codex with Claude approval
Risk: Low
Behavior change: None

Files

Work

  1. Add schema constants for logical output, output-attempt, playback, evidence stage, terminal outcome, and confidence while preserving existing static playback fields.
  2. Add bounded model_request_id, request_kind, and output_expectation=required|allowed|forbidden|unknown fields.
  3. Freeze the additive raw events exactly as model_request_recorded, model_output_attempt_started, and model_output_attempt_terminal; the last carries a typed outcome rather than separate provider-lifecycle event names.
  4. Extend record_playback_event validation to accept the new optional fields.
  5. Do not change existing event output when optional fields are absent.
  6. Build a synthetic, PII-free ledger representing:
    • matched caller input;
    • consent state entry;
    • model attempt 1 start and interruption;
    • model attempt 2 linked restart with matching prefix;
    • one browser playback bound to attempt 2;
    • model-received PCM followed by a failed outbound send;
    • completed second attempt;
    • a separate pair of identical static outputs requested by two FSM visits;
    • one transport duplicate represented by duplicate starts/copies for the same playback ID;
    • distinct playback IDs for one output attempt with no retry relationship, producing a weaker unexplained-multiple finding;
    • a policy request with forbidden output followed by an unexpected model response.
  7. Commit the expected normalized attempt rows and findings only after Claude approves the reconciled schema in CL2C-002.

Tests

Done

One schema commit; no imports from riff/live/session.py; no UI changes.

AC-01: Request Identity and Structural Gates

Owner: Claude
Risk: Medium
Behavior change: None
Feature flag: RIFF_DEBUG_OUTPUT_ATTEMPTS=1 until validated

Files

Work

  1. Freeze a baseline inventory of all 10 caller-facing direct send_client_content calls in riff/live/session.py and all 11 direct self.turn.reset() calls. Record each call's semantic reason in the packet result.
  2. Add _send_client_content_with_evidence(...), the only approved caller in GeminiLiveSession. It delegates exactly one unchanged client call and does not add an await, retry, copy, or reorder.
  3. With the flag on, allocate a bounded model_request_id from the semantic key (session_id, causal_visit_id, request_ordinal). The encoded ID uses stable ordinals/tokens and must pass the existing 160-character safe-ID validator; wall time and content equality are prohibited.
  4. Require every call site to supply request_kind and output_expectation=required|allowed|forbidden|unknown. Allocate a logical output at request time only for required/allowed speech.
  5. Emit model_request_recorded(dispatch_outcome=accepted) only after the awaited provider call returns. On its existing exception path, emit dispatch_outcome=failed best-effort and re-raise the identical exception; never retry or translate it.
  6. Bind input_id, selected_source, and decision_reason when the request is caused by a caller decision. Reference the full transcript_sources event; do not duplicate all transcript text.
  7. Replace all 10 session send calls with the wrapper. Every call site supplies a bounded source_kind and typed request reason. Do not wrap the separate inbound caller-text send in riff/live/transport.py as output.
  8. Add _reset_turn_scratch(reason). The output-attempt tracker lives on GeminiLiveSession, not TurnAccumulator; the wrapper records an attempt boundary or terminal-unavailable marker before calling self.turn.reset().
  9. Replace all 11 raw resets with the wrapper and a typed reason.
  10. Add source gates that reject a new direct send or raw reset outside its wrapper. The tests must prove each gate fails on a violating source fixture.

Tests

Done

Claude appends the two baseline inventories, event examples, flag-off proof, and test output to CL2C-*. Codex confirms the identity spine before AC-02.

AC-02: Provider Output-Attempt Capture

Owner: Claude
Risk: High telemetry risk, zero intended behavior risk
Feature flag: RIFF_DEBUG_OUTPUT_ATTEMPTS=1 until validated

Files

Work

  1. Extend _translate_server_message to emit provider interruption before turn_complete in the same message and retain a provider response ID when the SDK exposes one.
  2. If no stable provider ID exists, allocate a connection-local monotonic response ordinal labelled derived_local; never present it as provider ID.
  3. Open the first output attempt on the first output transcription or PCM tied to a logical request. Two PCM chunks remain one attempt.
  4. If output arrives for a request with output_expectation=forbidden, allocate an observed-unexpected logical output linked to that request. Do not relabel it as intended.
  5. For a logical output with an existing started attempt, open another only after an explicit interruption, changed response ID, typed RIFF retry/resume, or _reset_turn_scratch(reason) followed by fresh first text/PCM.
  6. When the provider bit is absent and a recorded boundary precedes fresh output, close the prior attempt as derived interruption and link the new attempt at lower confidence. A time gap or repeated text alone is forbidden.
  7. record_audio_out records model_received bytes and extends assistant_first_audio with causal IDs. It must not emit outbound/client/carrier playback truth.
  8. Terminalize the provider attempt on explicit interruption, turn_complete, typed provider failure, or teardown. An unclassified reset records OUTPUT_ATTEMPT_TERMINAL_UNAVAILABLE; it never synthesizes completion.
  9. Preserve the prior assistant_first_audio fields for compatibility and keep the attempt tracker independent of TurnAccumulator scratch fields.

Tests

Done

No caller-facing text/output ordering delta in fake-client transcript and PCM goldens. Codex reviews the raw attempt artifact before playback capture begins.

AC-03A: Browser Playback Truth

Owner: Claude
Risk: Medium protocol/telemetry risk, zero intended audio-content risk
Feature flag: RIFF_DEBUG_OUTPUT_ATTEMPTS=1

Files

Work

  1. Bind each generated binary PCM stream to a playback_id and its output_attempt_id. Send one bounded control envelope before its first PCM frame and one terminal boundary; never add metadata to every frame.
  2. Preserve every PCM byte and binary-frame order. Existing clients with the flag off receive the byte-identical stream and no new control envelope.
  3. Emit audio_requested when dispatch begins. Emit an outbound_accepted acknowledgement only after successful await ws.send(payload); a failed send cannot appear accepted.
  4. Extend the browser player/worklet to report client_playback start when the first sample for that playback ID is consumed, completion after the explicit end marker drains, interruption on clear/stop/supersession, and typed error.
  5. Persist client acknowledgements through the existing playback_event endpoint with logical/output/playback IDs and evidence stage.
  6. Do not claim browser speaker perception; the acknowledgement proves the Web Audio boundary only.

Tests

Done

One browser call can distinguish model receipt, server WebSocket acceptance, and actual Web Audio start/terminal on the same playback ID.

AC-03B: Phone Outbound Truth

Owner: Claude
Risk: Medium telemetry risk, zero intended audio-content risk
Feature flag: RIFF_DEBUG_OUTPUT_ATTEMPTS=1

Files

Work

  1. Allocate one playback_id for each phone delivery of an output attempt and retain it across all PCM chunks for that attempt.
  2. Emit audio_requested before the first encoded chunk. Emit outbound_accepted start only after the first successful ws_send inside _emit_pcm24_bytes_locked.
  3. Accumulate exact submitted wire bytes and the existing audio_agent_played clock interval without emitting one lifecycle row per chunk.
  4. Terminalize through a nonblocking transport observer when AC-02 reports the provider attempt terminal: completed after all submitted chunks, interrupted on barge/clear/supersession, or failed when no chunk succeeds.
  5. Preserve current swallowed-exception behavior and audio ordering. Telemetry observes ws_send; it does not change whether an exception escapes.
  6. Label this stage RIFF outbound acceptance. Carrier audibility remains the separately authorized dual-channel recording contract.

Tests

Done

A phone artifact distinguishes model receipt from submitted Telnyx wire audio; it still labels carrier confirmation unavailable unless independently retained.

AC-04: Capture Behavior-Affecting Context

Owner: Claude/service lane
Risk: Medium
Behavior change: None

Files

Work

  1. Emit state_context_snapshot at the same entry/exit boundaries as state_slot_snapshot.
  2. Record at minimum turn_count, no_progress_streak, max_no_progress_loops, decision_epoch, bounded recent states, pending transition identity, and active logical-output/output-attempt/playback IDs.
  3. Inventory unavailable behavior-affecting fields rather than silently omitting them.
  4. Extend guard telemetry with context_reads. For stalled, record the exact no_progress_streak and max_no_progress_loops values used.
  5. Keep context reads distinct from slot reads.
  6. Never mutate context while serializing it.

Tests

Done

The motivating stalled question can be answered from a synthetic ledger without reading current Python.

AC-05: Normalize Output and Playback Attempts

Owner: Codex
Risk: Medium
Behavior change: None

Files

Work

  1. Add pure _build_output_attempts over recorded request/provider evidence and _build_playback_attempts over canonical playback lifecycle events.
  2. Join request -> output attempt -> playback ID by explicit IDs; never by nearest timestamp alone.
  3. Validate provider terminals separately from playback-stage terminals.
  4. Preserve causal and observed visits separately.
  5. Preserve selected_source and decision_reason on compact input links while retaining source references to full transcript observations.
  6. Derive compatibility playback_spans from normalized playback attempts.
  7. Add typed findings for missing/contradictory lifecycle and unavailable stage.
  8. Fold events only in (seq, source_suborder) order. No mutable flow file, clock, random value, process hash order, or set iteration may affect output.

Tests

Done

Artifact schema version increments; migration/legacy behavior is documented.

AC-06: Mechanism-Aware Findings and Pause Audit

Owner: Codex
Risk: Medium

Files

Work

Implement only evidence-backed findings:

Every finding includes required source event IDs, causal visit, confidence class, and typed unavailable dependencies. Keep legacy DUPLICATE_AGENT_PLAYBACK for pre-schema calls.

Tests

Done

The synthetic motivating fixture produces the expected distinct findings and zero false transport-duplicate claim.

AC-07: Compact explain_moment and AI CLI

Owner: Codex
Risk: Medium

Files

Work

  1. Add explain_moment with exactly one event/time anchor.
  2. Resolve state, causal input, selected STT and decision reason, context reads, model-request kind/expectation, output attempts, playback attempts/stages, first deterministic divergence, gaps, and next commands.
  3. Default to a canonical response of at most 4 KB.
  4. Add detail=evidence with a 32 KB cap and pagination.
  5. Return raw events only via filtered, explicit references.
  6. Add latest and explain-moment ergonomic CLI commands without duplicating engine logic.
  7. Build deterministic indexes by event, visit, input, logical output, output attempt, and playback ID so each join is bounded by IDs plus the requested time window rather than a full-timeline scan.
  8. Include zero-model-call execution metadata.

Tests

Done

Claude can diagnose the synthetic failure using only latest, explain-moment, and one optional detail command.

AC-08: Browser Causal UI

Owner: Codex
Risk: Medium

Files

Work

  1. Render one Agent-lane band per playback attempt, grouped by logical output and output attempt, with textual stage and outcome.
  2. Connect linked output restarts and label transport retries without adding another cursor.
  3. Add Explain this moment to the existing inspector.
  4. Render a downward causal story: caller -> selected STT/reason -> decision -> state -> model request/expectation -> output attempt -> playback stage -> interruption/restart -> finding.
  5. Keep Entry/Exit slots and context values visible in the right inspector.
  6. Store attempt IDs and finding code with durable issue markers.
  7. Add a bounded Replay this evidence interval and Confirm this point action. An AI may propose a marker/deep link; only the operator confirmation changes its status from proposed to confirmed.
  8. Preserve mobile ordering and no horizontal overflow.

Tests

Done

An operator can locate and share the repeated prefix without inspecting JSON.

AC-09: Reproduction and Regression Promotion Seam

Owner: Codex design; implementation only after separate approval
Risk: High

Work

  1. Add logical outputs, output/playback attempts, truth stages, selected input, and context snapshots to exported capsules.
  2. Define a PII-safe synthetic input fixture for the marked defect.
  3. Define, but do not yet enable, an isolated runner request containing exact engine/flow revisions and typed overlays.
  4. Define promotion output: a deterministic regression fixture and expected finding/exit/audio-attempt assertions.

Done

The packet produces a reviewed runner contract. The browser Fork button remains disabled.

AC-10: Validation and Rollout

Owner: Shared
Risk: Medium; OTA step requires human GO

Ladder

  1. Unit schemas and invariants.
  2. Synthetic artifact and debugger fixture.
  3. Fake Live client interruption/restart.
  4. Browser dynamic-playback lifecycle with aligned replay.
  5. Local fake Telnyx ws_send lifecycle and wire recording.
  6. Local retained-call rehydration.
  7. Feature flag on/off byte and pre-existing event comparison.
  8. One Telnyx OTA call after explicit human GO.

OTA Acceptance

Commit Plan

Use these commit boundaries unless the reconciled mailbox approval changes them:

test(debugger): freeze output-attempt schema fixture
refactor(live): centralize output request and scratch reset boundaries
feat(live): retain provider output attempts
feat(browser): retain generated playback acknowledgements
feat(phone): retain generated outbound playback lifecycle
feat(fsm): retain behavior-affecting state context
feat(debugger): normalize output and playback attempts
feat(debugger): classify output causality
feat(debugger): add bounded explain-moment projection
feat(debugger-ui): visualize output attempts and causal chain
docs(debugger): publish audio-causality operator guide

Every fix commit references its FX entry. Pure design/test fixture commits may reference the design document instead.

Packet Result Template

Every implementation result appended to the mailbox contains:

packet/task_id:
status: done | blocked | needs-human-go
commit:
files changed:
commands/tests:
event examples and artifact path:
flag-off equivalence evidence:
known unavailable evidence:
rollback:
next owner/action:

Rollback

Implementation Stop Conditions

Stop and post blocked to the other mailbox when:

Plan Approval Gate

Implementation begins only after:

  1. Claude's CL2C-001 required changes are incorporated.
  2. Codex appends C2C-002 with a resolution table.
  3. Claude appends CL2C-002 approving the reconciled schema and packet ordering or identifies one bounded remaining blocker.
  4. AC-00 schema/fixture is committed before live capture code changes.