Skip to content

WI-H05: Approve LLM enrichment spend and volume

WI-H05: Approve LLM enrichment spend and volume

Problem

ADR-0017 uses Claude to derive mechanics, weight and player count from prose — the fields discovery runs on, and the ones no free structured source provides.

This is the first running cost beyond Apple’s $99, and docs/architecture.md §1 sets a £0 constraint. Agents must never trigger a paid run, so the decision sits with a human.

In scope

  • Run a pilot on a small sample (~50 games) and measure actual cost per game.
  • Decide the catalogue volume to enrich.
  • Decide the model tier and effort level, deliberately left open in ADR-0017.
  • Set a hard cost ceiling per run.
  • Record the decision, and write a superseding ADR if it changes the §5 cost model.

Out of scope

  • Building WI-028. Development and testing against a handful of fixtures needs no approval — only paid full-catalogue runs do.
  • Approving any other spend. This covers enrichment only.

Definition of done

  • Pilot run completed on a small sample; measured cost per game recorded on the issue.
  • Extrapolated full-catalogue cost recorded, with the volume assumption stated.
  • Model tier and effort chosen, with the pilot numbers as justification.
  • Hard cost ceiling set and recorded.
  • Quality checked at the chosen tier — a cheaper tier that produces confidently-wrong mechanic tags is a false economy, per ADR-0017.
  • Decision recorded; superseding ADR written if the §5 cost model changes.

Verification

Cost figures on the issue come from an actual pilot run, not an estimate. That is the whole point of this item — ADR-0017 deliberately left the tier open so it could be decided against real numbers.

Notes

Three things make this cost smaller than it first appears, and they are worth confirming are actually in use before approving:

  • The Message Batches API halves it, and this workload has no latency requirement whatsoever.
  • Prompt caching of the shared controlled vocabulary means the per-game cost is only the unique prose.
  • It scales with catalogue size, not traffic, and is paid once per game rather than per request.

The quality check matters as much as the cost one. ADR-0017’s failure mode is a plausible, confidently-wrong tag — worse than a missing one, because it actively misleads and looks identical to a correct one in the UI. Check the confidence distribution and the review-queue rate at the chosen tier, not just the invoice.