Seed-first AI: serving LLM explanations at near-zero cost

Most “build an AI app” tutorials share the same shape: a request comes in, you call the LLM, you stream the answer back. That’s fine for a demo. It falls apart the moment you have (a) a fixed catalogue of things to explain and (b) real users who all ask about the same things.

ACT Learner is an AI study coach for Australia’s ACT road-rules test. Every question can have an AI-written explanation. The naive design — call the LLM on every “explain this” tap — has three problems:

• Latency. A good explanation is a few seconds of generation. Tapping why? and waiting 3–5 s, every single time, is a bad study loop.
• Cost. The question bank is finite, but traffic isn’t. You’d pay to regenerate the same explanation for the same question thousands of times.
• Non-determinism. Temperature > 0 means the explanation drifts on every call. Hard to QA, hard to trust.

The key observation: most explanations are user-agnostic. “Why is C correct for question 42?” has the same answer for everyone. So the real architecture isn’t “call the LLM” — it’s a cache hierarchy where the live model is the last resort, not the first.

Three tiers, LLM last

Every /api/explain request walks down three tiers and stops at the first hit:

1. Seed store — explanations pre-generated offline for every question, shipped as a static JSON file. A hit is a hash-map lookup: ~0 ms, $0, no network. Most requests stop here.
2. Per-user cache (Supabase) — for signed-in users who regenerate a personalised explanation (aware of the answer they actually picked), the result is cached per (user, question, correctness).
3. Live LLM — only when the seed is missing or the user forces a fresh, personalised generation. The result is written back to tier 2.

The runtime is almost boring, which is the point:

// src/app/api/explain/route.ts (condensed)
if (!force) {
  const seed = getSeedExplanation(questionId, language);
  if (seed) return json({ explanation: seed, cached: true, seed: true });
}
if (user && !force) {
  const cached = await supabase.from("cached_explanations")
    .select("explanation")
    .match({ user_id: user.id, question_id: questionId, language, user_correct })
    .maybeSingle();
  if (cached?.data) return json({ explanation: cached.data.explanation, cached: true });
}
// only now do we call the model
const explanation = await callLLM(buildPrompt(question, ...));

The interesting engineering isn’t the cascade — it’s making tier 1 safe.

The hard part: cached content goes stale silently

Pre-generated content has one nasty failure mode. You improve the prompt — tighter system instructions, a better example — and every one of your thousands of cached explanations is now the output of an old prompt. Nothing errors. Users just quietly get worse answers than your code implies.

ACT Learner solves this by making the prompt itself the cache key. Each seed records the hash of the exact prompt that produced it:

// src/lib/seed-explanation-prompt.ts
const promptHash = createHash("sha256")
  .update(systemPrompt)
  .update("\n---\n")
  .update(userPrompt)
  .digest("hex")
  .slice(0, 16);

A CI check recomputes the hash for every question from the current prompt and compares it to the stored one. If a single hash differs — because someone touched the system prompt, the handbook context, or the question text — the build fails until the affected seeds are regenerated. Stale content can’t ship.

That’s the whole trick: pre-generation gives you instant, free reads; the prompt-hash gate gives you back the freshness guarantee you’d otherwise lose. You get the latency of a static file with the correctness of “always generated by the latest prompt.”

Grounding: cite the handbook, not the model’s memory

Road-rules explanations have to be right, not just fluent. So the prompt isn’t just the question — it’s the question plus retrieved context from the official handbook:

const chunks = findRelevantChunks(question.category, question.prompt, 2);
// → "[Handbook  Giving Way > Roundabouts (p48-49)]  ..."

Two relevant chunks (with page numbers) are injected into every prompt, so the model explains from the source material and can cite pages — which both reduces hallucination and makes the answer verifiable. Crucially, the retrieved chunks are part of the hashed prompt. If the handbook context changes, the hash changes, and the CI gate forces regeneration. Grounding and freshness are the same mechanism.

Two details that matter in production

Correctness-aware cache keys. A wrong-answer explanation (“you fell for the most tempting trap, B…”) reads differently from a neutral one. The per-user cache key includes whether the user got it right, so a wrong attempt’s explanation never leaks into a later correct attempt.

Right model per job. Seeds are generated offline with Gemini-3-flash (empirically higher quality here, and cost is irrelevant offline). The rare live call uses an OpenAI-compatible endpoint. Both sit behind one retry-with-backoff interface, so a provider blip doesn’t fail a generation.

Measure everything. Every request — seed hit, cache hit, or live call — logs to an explain_requests table: status, whether the LLM was called, latency, token counts, the prompt hash, how many handbook chunks were used. That’s how you know your cache-hit rate and true per-explanation cost instead of guessing.

Takeaways

If you’re putting an LLM in front of users over a finite catalogue:

• Make the live call the exception, not the rule. Pre-generate what you can and serve it as static data.
• Treat the prompt as the cache key. Hash it, store the hash with the content, and fail the build when they diverge. That’s the difference between a cache and a liability.
• Ground, and hash the grounding too. Retrieved context belongs in the prompt — and therefore in the hash.
• Log every request, including the cache hits, or you’re flying blind on cost and quality.

The result: the vast majority of explanations are instant, free, and provably current, and the model is reserved for the long tail where it actually earns its latency. That’s the difference between a demo that calls an API and a product that ships.

ACT Learner is live.