← Choice Health 2026-07-09-architecture-rewrite-decision.md raw .md

RIFF boundary architecture: adopt contracts, evolve in place

Decision in one paragraph

Adopt the FSM / deterministic code / LLM boundary as a set of enforced contracts, but do not rewrite RIFF around a new event-bus FSM now. Evolve the current engine in place. The new work is larger than a voice: announce | model | none field: RIFF needs explicit narration ownership, a dynamic-announce building block, and a prompt delivery receipt that survives connect races, barge-in, and re-entry. Keep the existing guard-based control model unless evidence reaches the rewrite triggers in this document. Treat per-state LLM routing as a cascade/text/HTTP capability only; it does not affect the Gemini Live phone path and is not part of the voice rollout.

This is a decision about architecture direction. It is not approval to regenerate flows, deploy to a phone node, enable a feature flag, or place an OTA call. OTA still requires explicit human GO.

Decisions requested

Approve, reject, or amend these six decisions:

  1. Adopt the boundary contracts. The FSM owns legal progress; deterministic code owns resolution, validation, side-effect mediation, and delivery state; the LLM may interpret or phrase only inside declared envelopes.
  2. Evolve in place. Add contracts to the current compiler, state manager, live session, and transports instead of replacing the state graph or moving to an event-bus transition model.
  3. Build narration in two parts. First make ownership explicit and lintable; then add dynamic_announce plus delivery receipts. Ownership metadata alone is not the end state.
  4. Formalize one Interaction Loop. Give cascade and Live the same outcome names, progress rules, and trace vocabulary without pretending they share one model execution path today.
  5. Separate LLM-routing decisions. The existing per-state routing cascade may be enabled only for cascade/text/HTTP deployments after its own gates. Gemini Live voice routing requires a separate architecture decision.
  6. Decline a rewrite now, not forever. Reconsider only when the explicit triggers under "Rewrite reconsideration" are met.

Why this decision exists

The original architecture discussion bundled several different questions:

Those questions do not have one shared answer. Most control contracts already have an implementation foothold. The largest reliability gaps exposed by recent calls were narration, delivery, and transport gaps around that control layer. The correct response is to widen the contract boundary and enforce it, not to infer from a small incident sample that either the current FSM or a rewrite has been proven superior.

Scope and non-goals

In scope

Not in scope

Terms

Term Meaning in this document
Contract A declared invariant with a compile/load gate, runtime behavior, trace evidence, and a replay test. Documentation alone is not a contract.
Candidate An LLM or deterministic extractor's proposed value. It is not trusted context until validation succeeds.
Narration The words RIFF intends to speak and the declared component responsible for producing them.
Delivery The transport-observable lifecycle of intended audio: queued, started, progressed, completed, failed, or interrupted. It does not prove human perception.
Prompt receipt Runtime evidence that a specific prompt reached a defined delivery threshold. Never call this a "heard" receipt.
Interaction Loop Enter state, satisfy narration obligation, observe, resolve/extract, validate, account for progress, transition or retry, mediate actions, and emit trace evidence.
Rewrite Replacing the current guard/pull transition model or flow representation, not an incremental compiler emission change.

Architectural ownership

The system has six layers. A layer may consume facts from the layer below it, but it must not silently take over another layer's decision.

Layer Owns Must not own
Transport identity Binding webhook, call, stream, and media lifecycle to one call/business Business routing guesses from stale or unnamed streams
Audio delivery Playback/synthesis lifecycle, interruption fraction, completion/failure events Whether a branch is legal or a caller answer is valid
Interaction Loop Turn ordering, prompt obligations, attempt/progress accounting, trace vocabulary Business-specific branch policy
FSM/control Legal states, guards, transitions, terminal outcomes, escalation targets Free-form language interpretation
Deterministic capabilities Choice resolution, schema validation, side-effect mediation, idempotency Inventing missing business facts
LLM Candidate extraction and natural wording within a declared envelope Adding choices, choosing undeclared branches, committing side effects, or claiming unobserved success

The non-negotiable boundary

The LLM may propose; deterministic code validates; the FSM decides whether progress is legal; the transport reports what it delivered. No model utterance, tool call, or audio callback is itself proof that the state may advance.

Normative contract stack

The words MUST, MUST NOT, SHOULD, and MAY below are normative.

C1. Menu contract

Existing footholds: locked_choice, evaluate_choice, alias compilation, tests/test_choice_escape_coverage.py, and the FX-069 DTMF gate.

C2. Extraction contract

Existing footholds: typed slots, enum abstention, deterministic extractors, and all_state_slots_filled_and_valid.

C3. Branch contract

RIFF's pull-based guards satisfy this shape. Named pushed events are not required to obtain it.

C4. Validation and sole-exit contract

C5. Progress, retry, and re-entry contract

Existing footholds: no_progress_streak, stalled, state-stamped user text, and FX-076/078 recovery behavior. The missing piece is a uniform engine-level policy.

C6. Escalation contract

FX-080 demonstrates why "has an escape" is weaker than "has the correct escape."

C7. Narration ownership contract

Every compiled state MUST declare a narration policy. The canonical proposal is narration, not a scalar voice, because voice already names synthesis identity and field-level voice policies elsewhere in the pack schema.

narration:
  owner: static_announce      # static_announce | dynamic_announce | model | none
  obligation: required       # required | optional | forbidden
  prompt_id: main_menu_v3
  on_interrupt: replay_if_unmet  # replay_if_unmet | continue | escalate
  max_replays: 3
  on_replay_exhausted: escalate
  one_shot:
    min_delivery_fraction: 0.75
  answer_admission:
    requires_content_delivered: true

Rules:

The initial migration MAY derive policies to produce a report, but generated packs MUST materialize the final policy. Hidden compiler defaults are not acceptable as the steady state.

C8. Prompt delivery contract

Narration ownership answers who should produce words. It does not answer whether the audio survived. Delivery is a separate runtime contract.

A delivery receipt has this minimum shape:

{
  "prompt_id": "identity_confirmation_v1",
  "state_epoch": "replicant_confirm_identity:7",
  "state_id": "replicant_confirm_identity_announce",
  "source": "dynamic_announce",
  "status": "completed",
  "prompt_content_hash": "sha256:resolved-text-and-render-policy",
  "audio_content_hash": "sha256:audio-bytes-played",
  "delivered_fraction": 1.0,
  "delivered_through_word_index": 17,
  "required_content_delivered": true,
  "one_shot_consumable": true,
  "interrupted_by": null,
  "attempt": 1
}

Allowed terminal statuses are completed, interrupted, and failed. queued and started are non-terminal observations. The receipt means transport-observed delivery only; it MUST NOT be described as proof the caller heard or understood it.

Input hubs that depend on a prompt declare:

requires_prompt_receipt: identity_confirmation_v1

Rules:

This contract closes the gap that a voice: declaration alone leaves open.

C9. Transport identity contract

FX-075/079/080 show that a correct FSM cannot compensate for the wrong caller's stream.

C10. Side-effect and success contract

C11. Operator-visible evidence contract

The Interaction Loop

The Interaction Loop is a shared behavioral model, not a claim that cascade and Gemini Live have one implementation today.

For each state epoch:

  1. Enter with validated context and a unique state epoch.
  2. Satisfy narration or establish that narration is optional/forbidden.
  3. Observe caller input, DTMF, provider result, timeout, or delivery event.
  4. Resolve/extract deterministic signals first; ask an LLM only inside the declared capability envelope.
  5. Validate candidates and action preconditions.
  6. Account for progress, failed attempt, interruption, or no-op.
  7. Decide exactly one transition, retry, escalation, or wait outcome.
  8. Commit transition mutations and mediated side effects idempotently.
  9. Trace the inputs, outcome, owner, receipt, guard verdict, and resulting state.

The first implementation target is common typed outcomes and trace fields. It is not a big-bang refactor of turn.py and live/session.py. A new shared abstraction is justified only where it removes duplicated semantics and has parity tests on both paths.

Current implementation map

Contract Current footing Required delta
Menu locked_choice, aliases, deterministic resolver, escape tests One compiler-owned announced projection, full contract lint, and a Phase 1 parity fixture proving cascade and Live dispatchers return identical outcomes for identical input
Extraction Typed slots, validators, enum abstention Explicit counted schema-failure outcomes and provenance
Branch Guard registry and transition evaluator Purity/serialization gate and cross-path sets/clears parity
Validation Required-slot success guards Typed validator outcomes and explicit precedence
Retry/re-entry no_progress_streak, stalled, state-stamped input Per-state policy and engine-level idempotent re-entry
Escalation Per-kind edges after FX-076/080 Exhaustive compile gate and run-record outcome
Narration Suppression lists, announce/hub idiom, voiceless lint Declarative policy on every generated state; all-path analysis
Delivery Audio events and barge timing exist in transports Prompt identity, receipts, common semantics, hub precondition
Transport identity FX-075/079 binding fixes and tripwire Durable identity invariant and overlap replay suite
Side effects State manager and required tools Uniform idempotency/success contract across capabilities
Operator evidence Turn/run records and nested debug payloads Promote all contract outcomes into stable operator-visible fields

Evidence matrix

Recent incidents are coverage evidence, not votes. They overlap, some share one call, and FX-080 was found by adversarial review rather than by a live caller. Therefore no "6.5 of 8" conclusion is used.

Evidence Primary contract Would this architecture help? Current status
FX-069: unannounced digits reached hidden choices C1 Menu Prevent at compile/test and resolver Fixed with generalized DTMF gate
FX-071: negated message intent became affirmative C2 Extraction Reject candidate or abstain Fixed with negation handling
FX-075: stale call captured another stream C9 Transport identity Detect/prevent before FSM input Fixed and hardened by FX-079/080
Property call: muted choice state had no speaker C7 Narration Compile failure on owner/path gap Stop-gap unmute and lint shipped
FX-077: state entered before Live connected C7/C8 Narration/delivery Require connect-time prompt receipt before answer Narrow locked-choice fix shipped; general contract open
FX-078: early barge burned one-shot menu C5/C8 Re-entry/delivery Commit one-shot from delivery outcome Barge path mitigated; zero-delivery audio failure/missing path remains open until separately fixed and proven
FX-076: caller trapped in message collection C5/C6 Retry/escalation Require bounded declared exhaustion Fixed with collect escape gate
FX-080: emergency stall returned to menu C6 Escalation Require correct typed target, not any escape Fixed through adversarial review

FX-074 is migration evidence, not a caller-failure vote: rebuilding two stale flows produced hundreds of lines of unrelated spoken-copy drift. That proves regeneration needs a controlled diff/audio/deploy process. It does not measure the engineering cost of an event-FSM rewrite, and this document does not claim that it does.

Execution-path truth: LLM routing

The repository has two materially different model execution paths:

  1. Cascade/text/HTTP: run_turn selects an adapter. The per-state policy cascade and RIFF_PRIMARY_LLM_ROUTING can affect this path. A repeat-N text-mode result measured an approximately 85% cost reduction for a scoped coffee-flow assignment.
  2. Gemini Live voice: GeminiLiveSession opens one persistent model connection for the call. It does not invoke the cascade's per-state adapter selector.

Therefore:

Alternatives considered

Option Reliability benefit Migration/replay risk Decision
A. Rewrite flows and control around event-style lifecycle objects Could unify event provenance if designed well Highest migration surface; re-proves mature guards, compiler output, audio, and all flows Decline now; retain revisit triggers
B. Add the full contract stack to the existing engine Directly covers observed control, narration, delivery, and identity failures Incremental but spans compiler, Live, and transport; requires strict phase gates Choose
C. Keep fixes as tests and conventions only Lowest immediate cost Hidden defaults and path-specific patches continue to drift Decline

Why option B

Delivery plan and gates

No phase starts implementation until Fable 5 review findings are resolved and the operator approves the amended design.

Phase 0 - freeze the decision and baseline

Deliverables

Exit gate: evidence can be regenerated from a clean checkout without an OTA call.

Phase 1 - enforce existing control contracts

Deliverables

Exit gate: all packs cold-compile; all routed flows pass focused contract tests; removing each new gate makes its reproduction fail. The C9 overlap, stale-entry, and missing-hangup replays pass; removing serialization or atomic binding makes the FX-075/079 reproductions fail. The locked-choice parity fixture runs both dispatchers, not only cascade.

Rollback: remove additive gates before any flow regeneration. No live behavior changes in this phase.

Phase 2 - materialize narration ownership

Deliverables

Exit gate: every state has one valid policy; every required prompt has an owner; no suppressed state is model-owned; every none input hub proves its prompt source on every reachable inbound path; the Phase 0 exposure counts are zero or have an explicit migration in the operator-approved compatibility report. A conformance replay per routed flow compares each declared owner with the observed narration source per state epoch; any mismatch fails.

Rollback: compiled metadata is additive and ignored by runtime until Phase 3.

Phase 3 - build dynamic_announce and prompt receipts

Deliverables

Exit gate

Rollback: runtime feature flag returns affected states to the last approved owner before regeneration. After regeneration, rollback means restoring the prior generated flow/audio artifact as defined in Phase 5; a flag cannot restore an owner that regeneration removed.

Phase 4 - formalize Interaction Loop parity

Deliverables

Exit gate: one replay fixture produces equivalent control outcomes and required trace evidence on both paths, with documented differences limited to model/audio execution. Remaining differences live in an explicit allowlist; every entry has an owner and removal condition, and an undocumented divergence fails the gate.

Rollback: keep the old path behind a temporary comparison flag until parity is demonstrated; do not delete working rescue behavior first.

Phase 5 - regenerate and promote one business at a time

Order:

  1. Property management first because it exercises per-caller identity and dynamic ticket narration. It is also the highest-risk pilot: its stop-gap unmutes and model-voiced locked-choice states are load-bearing. Its Phase 2 report MUST show each affected state resolving deliberately to dynamic_announce or another approved owner, never silently defaulting to static_announce.
  2. One low-risk static-menu business.
  3. Remaining businesses only after both shapes promote.

For each business:

Rollback: drain candidate, restore the prior flow/audio manifest and code SHA, restart, and verify health before accepting calls.

Separate track - cascade/text/HTTP LLM routing

This track is not a prerequisite for Phases 0-5 and does not claim a voice win.

Definition of done

The architecture is adopted only when all of these are true:

Language and compiler

Runtime

Evidence

Operations

Ownership

Work Owner
Compiler, FSM, Live session, providers, service traces, focused tests M3 service lane
Caller harness, replay bundles, promotion reports, OTA artifacts M5 client lane
Spoken-copy/audio review and explicit OTA GO Human operator
Architecture approval and unresolved policy choices Human operator after Fable 5/Codex review

Cross-lane findings should be posted with evidence rather than repaired by crossing ownership boundaries casually.

Risks and mitigations

Risk Consequence Mitigation/gate
Ownership metadata creates false confidence Declared owner still produces silence Separate C7 ownership from C8 delivery receipt
dynamic_announce adds latency or fails Dead air before input hub Phase 0 baseline, deterministic fallback, measured Phase 3 gate
Dynamic synthesis persists caller data PII audio enters git/shared manifests or fills node storage Call-scoped artifacts, versioned-manifest exclusion, retention bound, reachability test
Receipt refers to stale audio Full delivery is recorded for an obsolete prompt asset Bind prompt and played-audio hashes; fail critical admission on missing/mismatched binding
Static and dynamic voices sound inconsistent Caller hears multiple personas One prompt owner per epoch; voice/model identity pinned in trace and audio config
Live and cascade drift Tests pass in text but fail by phone Common outcome/trace contract plus path-specific replays
Compiler regeneration changes copy Audio churn and unreviewed caller behavior Per-business diff, audio manifest gate, candidate rollout
Receipt is mislabeled as human perception Unsafe confirmation inference Call it delivery receipt; exact threshold and limitations in schema/trace
One fraction is reused for two decisions A truncated confirmation admits yes before its critical value is spoken Separate one_shot_consumable from semantic required_content_delivered
Retry fixes demote safety escalation Emergency caller returns to menu Typed exhaustion target and adversarial escape-target test
Compatibility flag becomes permanent Two architectures accumulate Every flag has removal criteria and an owner in the implementation plan

Rewrite reconsideration

"Decline rewrite" is the current decision, not a permanent ban. Open a new ADR when one or more of these is demonstrated with traces and reproductions:

  1. A required transition cannot be represented as a pure guard over validated context and explicit engine facts.
  2. Replay cannot reproduce ordering because causally relevant events are absent from the current state/transition record.
  3. Known hand-copied control semantics between cascade and Live, including the duplicated locked-choice dispatchers, persist or grow after Phase 1 parity and Phase 4 shared outcome/trace work, and parity tests cannot constrain them.
  4. Idempotent asynchronous actions require durable event processing that the current transition commit cannot provide.
  5. Flow/state growth causes a measured regression in cold-compile wall time, per-flow gate runtime, or defect escape rate past the contract gates that compiler composition demonstrably cannot control.
  6. Two or more promoted production defects share one architectural cause that the contract stack detects but cannot prevent without replacing the control model.

Meeting a trigger starts comparative design work; it does not automatically select an event bus. The replacement must show migration, replay, rollback, and call-quality advantages against the contract-based current engine.

Reviewer recommendations pending operator ratification

Fable 5 answered the eight v3 open decisions as follows. These become final only when the operator approves this ADR:

  1. dynamic_announce is a distinct compiled state type, not a render mode.
  2. Epoch-bound requires_prompt_receipt is sufficient for v1; introduce a larger handshake object only when a hub legitimately depends on multiple prompts.
  3. Only exact DTMF and exact spoken aliases of declared choices resolve during an announce; the hub performs the resolution from forwarded input.
  4. Model ownership remains legal for non-critical flexible narration. Critical prompts move to dynamic_announce; model ownership is only a flagged stop-gap.
  5. Critical content includes compiler-derived interpolation spans plus explicitly authored semantic markers for critical literal copy.
  6. Phase 0 sets numeric latency/failure limits from measurements. The rule form is fixed now: compare dynamic p95 time-to-first-audio with measured static p95 plus an approved budget, and require an explicit fallback on any breach.
  7. Shared outcome vocabulary before shared implementation is correct, except the known locked-choice dispatcher pair receives parity coverage in Phase 1.
  8. Rewrite triggers remain conditional; triggers 3 and 5 use the measurable forms in the amended "Rewrite reconsideration" section.

Repository evidence

The ADR is self-contained, but these versioned sources contain the detailed traces, mechanisms, and measurement qualifications behind it:


Paste-ready final Fable 5 verification packet

Paste this entire Markdown document, including the prompt below, into Claude Fable 5.

Review prompt

You are Claude Fable 5 acting as RIFF's senior adversarial architecture reviewer. RIFF is a production voice agent whose standard is: a real caller can complete a phone workflow, and every failure can be replayed, diagnosed, fixed, and promoted through deterministic, model-backed, service, and OTA testing.

This is final verification of the v4 ADR. Confirm that the round-two P1-P11 changes are present and enforceable: Phase 1 owns C9; critical model narration fails closed; receipts bind prompt, played audio, and state epoch; dynamic artifacts are call-scoped and retention-bounded; both locked-choice dispatchers have Phase 1 parity coverage; delivery replay is bounded; early input is forwarded to the hub; ambiguous provider outcomes do not retry unsafely; narration policy is checked against observed source; and Phase 4/rewrite gates use explicit measures and allowlists.

Do not optimize for agreement, politeness, or preserving the author's framing. Look for unsafe assumptions, contracts that cannot be enforced, execution-path confusion, hidden state, non-idempotent behavior, incomplete rollback, and plans whose tests can pass while a caller still fails.

Pay special attention to these counterexamples:

Return exactly these sections:

  1. Verdict: APPROVE, APPROVE WITH REQUIRED CHANGES, or REJECT.
  2. Findings: severity-ranked (Critical, High, Medium, Low), each citing the ADR section, the failure mechanism, and the smallest adequate correction.
  3. Invalid or unproven assumptions: distinguish facts from hypotheses.
  4. Contract counterexamples: give at least one concrete trace for every contract you believe is underspecified.
  5. Phase-gate audit: identify gates that are subjective, circular, non-vacuous only on paper, or unable to fail on the bad behavior.
  6. Decision audit: challenge the eight reviewer recommendations and identify any one the operator should reject.
  7. Minimal safe v1: identify the smallest independently shippable subset and what must explicitly wait.
  8. Rewrite ruling: say whether evolving in place remains justified, and name the evidence that would change your ruling.
  9. Proposed patch: provide replacement text for every required ADR change, not just commentary.

Do not credit a declaration as enforcement unless you can name its compile/load gate, runtime behavior, trace evidence, and reproduction test. Do not describe transport delivery as proof a human heard audio. Do not treat text-eval model-routing evidence as a Gemini Live voice result.