The Preflow Pattern — Prewarming Data, LLM Content, and Audio Before a Call

Date: 2026-06-30 Status: Pattern in production (reference: the Space Channel flow). Audience: RIFF engineers applying the preflow pattern to other flows.

Authoring note: this doc was scoped for Gemini/antigravity (agy), but agy hung in --print mode (a known issue), so it was written directly from the same brief. The content is the deliverable; the tool is interchangeable.

In plain terms

RIFF runs AI phone agents. A fixed state machine (the "flow") keeps the conversation on rails; an LLM decides what to say within each state. On a live phone call, latency is everything — every second of silence while the agent "thinks" is dead air the caller hears.

A preflow is a job that runs before any call and prepares everything the call will need, so the live flow only has to read cached data and play pre-rendered audio. Nothing slow happens while the caller waits.

The big idea, in one line: do the slow, expensive work (web fetches, LLM writing, speech synthesis) ahead of time, on a schedule — not mid-call.

The problem it solves

While a caller is on the line, the flow must never:

• scrape a website or call a slow external API,
• run an LLM to write prose,
• synthesize speech from scratch.

Any of these creates dead air and unpredictable quality. A phone call has no spinner and no undo.

The idea: three stages

A preflow runs on a schedule (e.g. every 12 hours) and does three things:

   ┌─ 1. DATA REFRESH ─┐   ┌─ 2. LLM CONTENT ──┐   ┌─ 3. AUDIO RENDER ─┐
   │ fetch external →  │ → │ raw data → spoken │ → │ text → TTS WAVs   │
   │ JSON snapshot     │   │ content (offline) │   │ in an audio cache │
   └───────────────────┘   └───────────────────┘   └───────────────────┘

1. Data refresh. Fetch external data into a small JSON snapshot with a schema and an expires_at/TTL. Space Channel example: upcoming rocket launches, space news, NOAA solar weather, UFO wire items, Deep Space Network status.

2. LLM content generation. At build time (not call time), turn the raw snapshot into the actual spoken content. Space Channel example: an integrity-checked, anchor-style "3 headlines + summary" brief per topic, written by an LLM. Because this runs offline, it can be slower and higher-quality, and it's parallelized across items.

3. Audio render. Pre-render the static and data-resolved lines to a TTS audio cache, keyed by (text, voice, model). At call time a cache hit means the audio is served instantly from disk.

At call time

The flow reads the fresh snapshot (a content tool returns the pre-built summary) and plays the matching pre-rendered WAV. The caller gets an instant, predetermined, high-quality briefing.

No server restart is needed when a new snapshot is written — the flow reads the snapshot per call and serves WAVs from disk by content hash.

Freshness and the TTL gap (read this twice)

Each snapshot stores generated_at, expires_at, and a TTL. A loader returns the cache if fresh, otherwise refreshes.

The subtle trap: the live call path refreshes without the LLM — the turn loop has no LLM budget for batch generation. So if a briefed snapshot expires, the next call regenerates plain, non-LLM copy (it silently degrades from "anchor brief" to "catalog summary").

The fix: a scheduled briefed-refresh whose cadence is shorter than the TTL, so the snapshot is never stale when a call arrives. In Space Channel:

• a launchd job runs the briefed preflow every 12 hours, and
• the snapshot TTL is 14 hours — a 2-hour buffer so it never expires into a gap.

Design seams worth copying

These are the lessons that make the pattern safe and reusable:

• Opt-in LLM via an injected callable. The content builder takes an optional headline_fn (story -> ranked headlines). With it, summaries become rich LLM briefs; without it, deterministic catalog summaries are used. This keeps the live and test paths deterministic and makes the LLM a pure build-time enhancer, not a runtime dependency.
• Fail-closed. If the LLM or its integrity gate drops everything, fall back to grounded source text — never invent content on a phone line.
• Parallelize the offline LLM calls (thread pool + per-item timeout). Serial was ~16 s/item × ~12 items = 20+ minutes and stalled on any slow call; fanned out it's ~40–75 seconds.
• Provider choice is an ops decision. Offline briefs use a fast batch model (DeepSeek/Qwen), selected by env or a flow-level class. The live voice turn keeps a native realtime-audio model (Gemini Live). Different jobs, different tools — don't conflate them.

Reference implementation (Space Channel)

Why this generalizes (the vision)

Any flow that depends on external data or expensive LLM content can have a preflow. The preflow prepopulates data so the LLM can pre-build conversation content/scenarios ahead of time — making calls faster and more consistent for all flows, not just Space Channel.

Checklist: adding a preflow to a new flow

1. Identify the slow/external inputs the flow needs (APIs, scraped data, LLM-written prose, synthesized speech).
2. Define a snapshot schema + TTL. What JSON does a call read? How long is it valid?
3. Write a build-time content generator with:
   • an opt-in LLM seam (inject the model as a callable),
   • a fail-closed fallback to grounded source text.
4. Pre-render audio for static and data-resolved lines into the shared audio cache (keyed by text+voice+model).
5. Schedule a refresh whose cadence < TTL (cron/launchd), so callers never hit a stale snapshot.
6. Verify with an end-to-end call simulation across every subpath before trusting it live.

Cost footnote

Because the preflow pre-renders the agent's speech (local TTS) and pre-generates content offline (cheap batch model), the live call pushes almost all cost onto the cheap input side (understanding the caller). Pre-rendering agent audio instead of generating it live is roughly a 10× cost reduction per talk-hour — the preflow is a cost story as much as a latency one.