RIFF — Remote Control Design, v0 (control events + surface schema)
Status: design handoff for the implementing Claude — open decisions marked OD-Rn
Date: 2026-07-01
Depends on: riff-artifact-trace-schema-v0.md (artifact §3, trace §6, I1–I6), riff-console-map-design-v0.md (<KeypadChips>), riff-companion-channel-design-v0.md (link/token/stream, taps, receipt)
Reframe adopted: the companion page is not a Space Channel mirror — it is the second screen of a modality-agnostic call. A tap is a third input to the same FSM that voice and DTMF drive. This doc specifies the two missing pieces: the input mirror of the §6 trace contract (control events) and a compiler-emitted surface schema so every flow gets a remote control for free.
0. New invariants
- I7 — voice-complete. No state may require screen input. Every menu, form, and confirmation must be completable by voice/DTMF alone; the screen augments, never gates. Enforced at compile (§5). The companion can die mid-call (battery, tunnel, never opened) and the call proceeds unchanged. Future tension: payment-by-screen may want to violate this — the resolution is that such a state must declare a voice alternative (pay-on-arrival, agent handoff), not an exemption.
- I8 — one router. All input modalities normalize to a single
InputEventbefore arbitration. There is exactly one decision mechanism (locked_choice); adding a modality never adds a router.
1. The input contract — control_event
The symmetric twin of the trace: trace events flow engine → subscribers; control events flow subscribers → engine, over the same per-call WS, gated by the same token.
1.1 Wire envelope
{"v":"0.1","call_id":"c_9f21","role":"caller","channel":"companion",
"client_seq":3,"decision_epoch":14,"type":"tap",
"payload":{"node":"main_menu","token":"launches"}}
client_seq— per-client monotonic; the engine dedupes retries/reconnects.decision_epoch— the traceseqof thestate_enterthe client had rendered when the input was produced. This is the staleness and idempotency mechanism (generalizes the companion doc's staleness guard).
1.2 Event types (define all now; implement tap + submit first)
| type | payload | consumed by |
|---|---|---|
tap |
node, token |
choose decision points; do outcome selection |
submit |
node, fields: {slot: value} |
collect forms — atomic slot fill |
text |
node, slot, value |
typed free-text into collect (OD-R3, deferred = OD-C5) |
nav |
target: menu|back |
reloop affordances (compiles to a tap on the reloop token) |
There is no client→engine capability event: the one capability frame is
server→client, challenge_issued {code, expires} (§3.1). The screen never
claims anything — DTMF entry of the displayed code is the proof.
1.3 Normalization and arbitration — locked_choice unchanged
All modalities collapse to one internal event before any decision logic:
InputEvent {source: dtmf|voice|tap|submit, node, decision_epoch, resolved: token|fields, confidence}
- dtmf and tap:
confidence = 1.0, no grammar pass. voice: through constrained decode exactly as today, stamped with the node active at utterance start. - Control events enter the same single-consumer engine queue as transport events — the FSM turn loop is the serialization point; no new concurrency.
- Arbitration: first
InputEventthat is valid for the current decision epoch wins; the decision locks; every subsequent event for that epoch is dropped and traced. - Validity checks, in order: role/capability (§3) → epoch matches current decision point → token/fields in the node's grammar/slots. Failures emit
input_dropped {source, reason: capability|stale|locked|invalid}— the generalization ofcompanion_input_rejected, which this replaces.
The race, resolved: caller begins saying "one," taps "3" mid-utterance. The tap enqueues first (instant), wins, locks epoch 14, the FSM advances (a tap during audio is a barge_in via: web_tap). The utterance finishes decoding stamped with epoch 14, arrives at epoch 16, dropped stale, traced. Deterministic, no special case.
1.4 Trace deltas (amends §6 + companion §7)
route_decision.viaandbarge_in.viaaddweb_tap(already in companion doc) —submitappears ascollect_turn {via: submit}.- New:
input_dropped(above);capability_granted {tier, via: channel_bind}(§3). companion_input_rejectedis superseded byinput_dropped.
2. The surface schema — compiler-emitted
Every compiled node gains a surface block: the declarative projection of that state onto a second screen. riffc derives it; source YAML may override presentation (labels, grouping, order); L0 validates overrides against the grammar. Because the surface ships inside the artifact, the companion page is a generic renderer — one page, every flow, every tenant (theme = tenant config, the OD-M2 white-label hook).
2.1 Derivation rules
| node type | surface kind | derived from |
|---|---|---|
| choose | menu |
one button per grammar token, DTMF order, labels from token/aliases (override for display copy); reloop tokens render as nav |
| say | media |
text_ref as readable script, progress bar from duration_ms |
| collect | form |
one field per slot — requires typed slots (§2.2); enum slots → chips, tel/date/text → native widgets; submit button emits submit |
| do | progress → choice |
hold text while in-flight; enum outcome frames as buttons (a tap here is a tap on the frame's confirm choose) |
| end | receipt |
template + fields_from: [slots, tool_results] — this is the companion doc's §6 receipt, now just another surface kind |
| (reserved) | handoff |
continue-on-web (§4, out of v0) |
2.2 Required schema delta: extend slot_types (not a parallel system)
Slot typing already exists (riff/slot_types.py, Flow.slot_types) as extraction
semantics: {phone, time, name, email, address, date, quantity, text}. Schema 0.2
extends this facility rather than adding a second representation:
- New type:
enum, withoptions:. An enum is a value domain, not a widget: it constrains extraction (the extractor's target set is the options + null), validation, and the default widget (chips) in one declaration. Options accept scalar or object form:[water_heater, …]or{value: water_heater, label: "Water heater", aliases: [hot water, water heater]}. - New orthogonal presentation fields on any slot:
label:(display copy),optional: true,widget:(override; derived from type by default — enum→chips, phone→tel, date/time/email/quantity→native, others→text. Per OD-R5, an override may restyle, never add a control). reask:is the canonical per-slot VOICE prompt (resolves OD-R7). It already exists inslot_types; §4's form-timeout path ("the voice path prompts field-by-field") consumes exactly this.labelis what the screen shows;reaskis what the voice says — they may differ. L0-11 (§5) requires areaskon every non-optional slot of aformsurface.- No
kind:field, noteltype —phoneis the semantic;telis its widget.
slot_types:
job_type: {type: enum, options: [water_heater, drain, leak, other], label: "What's the problem?"}
urgency: {type: enum, options: [emergency, this_week, flexible]}
callback_number: {type: phone, label: "Callback number"}
notes: {type: text, optional: true}
Migration surface: only flows using bare required_slots without a slot_types
entry. Enum aliases feed the collect extractor as a constrained target set —
the same determinism move as constrained ASR, applied to slot filling.
Implementation note (this slice): the unified type: field is real — since
semantic names (enum, phone, …) never collide with JSON types (string,
integer, …), one type: field carries both; {semantic_type: …} long-form
still works. enum requires a non-empty options list (validated at load).
riffc L0-10 makes an untyped collect slot a hard error. Migration checklist:
docs/flow-eval/typed-slots-migration.md.
2.3 Example — main_menu compiled surface
main_menu:
type: choose
surface:
kind: menu
title: "Space Channel"
controls:
- {token: launches, label: "Launch status", accepts: [tap]}
- {token: news, label: "Space news", accepts: [tap]}
# …
- {token: goodbye, label: "End call", accepts: [tap], style: quiet}
3. Write capability — tiered, channel-bound
The unguessable link is a bearer read capability. Writing is tiered inside the token/session:
| tier | grants | granted by |
|---|---|---|
read |
the filtered trace stream | opening the link |
tap |
tap/nav/submit on non-committing states |
caller-role link + session pinning (possession of the SMS ≈ possession of the phone) — adequate for the booking threat model |
commit |
inputs on committing states (book, cancel, future payment) | channel-binding challenge (§3.1): the server shows a random code on the requesting session's screen; the caller enters it as DTMF. Knowledge of the code proves screen possession; the DTMF proves leg presence; together they prove the caller controls that browser session. |
- Which states are
commit-tier is declared in the artifact (surface.tier: commiton the confirm choose ofbook_it); default istap. - The challenge fires lazily — first commit-tier attempt only — so the happy path has zero added friction; the grant persists for the call.
- Enforcement is engine-side (§1.3 validity order) with per-session rate limits; every denial is an
input_dropped {reason: capability}in the trace. - v1 booking flows may reasonably mark nothing
commitand rely on pinnedtap(OD-R1); the mechanism exists in the schema because payment will mandate it and retrofitting authentication is miserable.
3.1 Channel-bind protocol (resolves OD-R6)
A fixed "press 7" fails even with single-pending semantics — the confused-caller attack: an attacker holding a leaked read-link taps commit, the prompt plays on the victim's call audio ("press 7 to confirm"), and a compliant caller presses 7 having tapped nothing, granting commit to the attacker. The prompt itself is the social-engineering vector. The binding must require something the phone-holder can only know by seeing the requesting session's screen — the device-pairing pattern:
- A commit-tier input arrives from a session lacking
commit. The engine holds the input, generates a challenge{code: 4 random digits, nonce, session_id, decision_epoch, ttl: 30s}, and enforces at most one pending challenge per call (a concurrent attempt from any other session →input_dropped {reason: challenge_pending}). - The code is sent only to the requesting WS session (
challenge_issued {code, expires}— server→client; the code never appears in the trace, which recordscapability_challenge_issued {session_ref}only). The FSM plays the compiled prompt: "to confirm, enter the code shown on your screen." - While the challenge is pending, the engine intercepts DTMF ahead of
locked_choice (the interrupted decision epoch is parked, not advanced).
Correct code →
capability_granted {tier: commit, session_ref, via: channel_bind}, and the held input auto-applies iff its decision_epoch is unchanged. Two wrong entries or TTL expiry → challenge void,input_dropped {reason: capability}, DTMF interception ends, the parked decision point resumes on the voice path (I7). - Scope note: the challenge gates screen-originated commits only. A spoken confirmation is inherently leg-bound and needs no challenge — I7 guarantees that voice path exists.
The confused-caller attack now fails safe: the victim hears "enter the code shown
on your screen," and their screen (if any) shows no code. This also deletes
challenge_ack — the screen never claims anything; DTMF entry of the correct
code is the proof.
4. Handoff patterns — the terminal surface generalized
The receipt was the first instance; the general pattern is the screen outliving or augmenting the voice turn, in both directions:
- Voice → screen (forms too long to speak): a collect whose surface is a form; the voice plays a compiled "I've sent a form to your screen — tap send when you're done, or just tell me." The FSM enters a bounded wait (I1:
max_turn_msstill applies) on eithersubmitor voice slot-fill — first to complete wins per the same arbitration; on timeout, the voice path prompts field-by-field. I7 is preserved because the voice path is mandatory. - Screen → voice: free — a tap is input; the FSM's next state speaks. Ambiguity resolution is just the existing confirm ladder.
- Screen death: no transition, no timeout, nothing — the voice path is the call (I7). The engine notices only via WS disconnect (
companion_closed), which affects nothing. - Call end → screen persists: the receipt (companion doc §6, unchanged: S3 unguessable key, 30-day lifecycle).
- Continue-on-web (
OD-R2, reserved):end {disposition: handoff_web}whose receipt is a live web session — this is where the companion page and voice.html converge (OD-C3). Schema reservessurface.kind: handoff; nothing built in v0.
5. L0 additions (compile-time)
- Every
surfacecontrol token exists in its node's grammar; every form field maps to a declared, typed slot; every receipt template field is resolvable from slots/tool_results. - I7 (resolves OD-R7): every collect state bearing a
formsurface must have (a) an entry voice prompt AND (b) areaskon every non-optional slot in that surface. Collects without a form surface are untouched — their voice path is their only path, covered by existing validation. This keeps the 0.2 migration surface identical to the slot-typing checklist (same flows), not doubled. - Every
commit-tier state is a choose (something lockable) with a compiled challenge prompt asset ("press 7…") present.
6. What already exists vs. what's new
| exists (design extensions against these) | new |
|---|---|
locked_choice deterministic router |
InputEvent normalization in front of it (I8) |
| §6 trace contract + per-call capability-scoped WS | control_event inbound on the same socket; input_dropped, capability_granted events |
riffc node classifier |
surface emission + typed slots (schema 0.2) + L0 rules 10–12 |
| companion page (follow-along live) | generic surface renderer replacing bespoke rendering |
| artifact model, receipt pipeline | capability tiers + channel-bind challenge |
7. Build order
Implementation status (2026-07-02):
- Step 1 typed slots — SHIPPED (
main): unifiedtype:+enum+date, enforced at compile (L0-10) + runtime; 228 tests, quality report indocs/flow-eval/strong-typing-quality-report.md. Surface emission + L0-11/12 in flight (parallel session; needsiter_slots+ OD-R7 rewrite). - Step 2 InputEvent — CONTRACT SHIPPED (
main,riff/input_event.py): theInputEvent+resolve_input_eventrouter (dtmf/voice→unchangedlocked_choice, tap→web_tapno-grammar), proven byte-identical to the direct evaluator across 1468 cases (decision-trace parity, robust to non-deterministic latency + async router scheduling). ADDITIVE — the live transport is not yet swapped onto it (that swap is the remaining part of Step 2). - Step 3 arbitration — SEMANTICS SHIPPED (
main,riff/input_arbiter.py):InputArbiter(I8 single serialization point) — first valid event for the opendecision_epochwins + locks; later same-epoch →locked, past →stale, no-match →invalid(no lock); rejects carry ainput_droppedframe. Acceptance A2 proven deterministically on the real hub (tap wins, slow voice droppedstale, exactly one route). ADDITIVE — not yet wired to the transport/WS. - Step 5 capability — SEMANTICS SHIPPED (
main,riff/capability.py): the §3.1 device-pairing channel-bind (resolves OD-R6) — tiered read/tap/commit; server issues a code to the requesting session only; DTMF on the leg grants commit to THAT session; one pending per call; void-after-attempts + TTL; the code never enters the trace. Acceptance A4 proven (9 tests: grant-only-requester, second-session-rejected, attacker-can't-be-granted, void/expiry, trace hygiene). ADDITIVE — not yet wired to transport/WS. - Remaining wiring (not started): the live-transport swap onto InputEvent+Arbiter;
tap/submit/challengeover the companion WS; the generic surface renderer (A1, needs the surface slice committed).
- Schema 0.2: typed slots +
surfaceblocks + trace deltas;riffcemits surfaces for both reference flows; L0 rules 10–12. (Pure compiler work, no runtime risk.) - InputEvent normalization: refactor DTMF + voice to produce
InputEvents feeding unchangedlocked_choice; property-test in the L1 sim that behavior is byte-identical to today (this is the risky refactor — do it before any web input exists so regressions are attributable). tapend-to-end: control_event over the existing WS, validity ladder,input_droppedtracing; the race test (§1.3) as an L1 fault-injection case.- Generic surface renderer on the companion page, replacing bespoke Space Channel rendering;
submitforms against austin_plumbing. - Capability tiers + channel-bind (may ship dormant per OD-R1).
Acceptance:
- A1: The same companion page, with zero page code changes, renders and drives both Space Channel and austin_plumbing purely from their artifacts.
- A2: The §1.3 race (speak "1", tap "3") yields exactly one route,
via: web_tap, with the voice input traced asinput_dropped {reason: stale}— verified in L1 sim and on a live call. - A3: Killing the companion page mid-call changes nothing on the voice leg (I7), and the trace shows only
companion_closed. - A4 (binding, not ritual): a commit-tier tap from session A triggers a code shown to A only; entering that code on the phone grants A and only A; session B (second tab or unpinned device) attempting commit while A's challenge is pending is rejected (
challenge_pending), and B can never be granted by A's code. (4 digits, 2 attempts, 30s.) - A5:
riffcrejects a surface button naming a nonexistent token, a form field with an untyped slot, and a screen-only state with no voice path (L0 rules 10–12), each with a distinct error.
8. Open decisions
| id | decision | default in v0 |
|---|---|---|
| OD-R1 | Commit-tier usage in v1 booking flows | Mechanism in schema; flows mark nothing commit initially — pinned tap suffices for the plumber threat model. Flip when cancel/payment lands. |
| OD-R2 | Continue-on-web handoff | surface.kind: handoff reserved; unbuilt. |
| OD-R3 | Typed free-text (text events) into collect |
Deferred (was OD-C5) — a fourth modality racing voice inside a turn has real design weight; submit (atomic form) covers the near-term need without per-keystroke arbitration. |
| OD-R4 | Voice+form mixed fill granularity | v0: submit is atomic and wins/loses whole; per-slot merge of voice-filled and form-filled fields is deferred until a real flow needs it. |
| OD-R5 | Surface overrides scope | Presentation only (labels, order, grouping, style). Overrides can never add a control without a grammar token — that would violate I7/I8 by the back door. |
9. Codex review — resolved (2026-07-02)
Codex reviewed this doc + the first implementation slice; §1.3 arbitration is sound (TOCTOU-safe: one FSM queue, decision locks before the next epoch). Both design findings are now resolved by Fable 5:
- OD-R6 — RESOLVED: device-pairing channel-bind (§3.1). A fixed "press 7"
fails even with single-pending semantics via the confused-caller attack (the
prompt plays on the victim's audio and is itself the social-engineering
vector). Resolution: the server shows a random code on the requesting
session's screen only; the caller enters it as DTMF. Knowledge proves screen
possession, DTMF proves leg presence — the conjunction proves control of that
session, and it fails safe.
challenge_ackis deleted (the code entry is the proof). See §3.1 + amended A4. - OD-R7 — RESOLVED: canonize
reask(option b).slot_types.reaskis the per-slot voice prompt (§4's form-timeout path already consumes it);labelis the screen copy,reaskthe voice. L0-11 (§5) is rewritten to fire only onform-surface collects, keeping the migration surface identical to the slot-typing checklist. See §2.2 + §5 rule 11.
Cross-cutting directive — one slot-enumeration contract. Codex's GoalSegment
finding (L0-10 + surface both read target_slots off StateDef, but GOAP target
slots live on GoalSegment) is fixed one level up: extract a single canonical
iter_slots(flow) that enumerates every slot-declaration site, and have
riffc, surface emission, AND the migration enumerator all consume it. Two
consumers independently walking slot sources is how this bug happened; three is
how it recurs at a new declaration site. Same "one contract" move as playback and
trace. Owned by the surface session (it owns riffc/surface/types).