Skip to content
Tabletop Tracker

ADR-0027: The catalogue is harvested, not hand-authored

ADR-0027: The catalogue is harvested, not hand-authored

  • Status: Accepted
  • Date: 2026-08-06
  • Supersedes: —
  • Superseded by: —

Context

ADR-0010 put the catalogue behind a CatalogueSource so that work could proceed before a real source existed, and WI-020 filled that seam with a hand-authored fixture. The fixture has never pretended to be the catalogue — its own docstring says so:

Roughly 200 real games, written from knowledge rather than harvested… WEIGHTS ARE APPROXIMATIONS and the mechanic tags are a pragmatic ad-hoc vocabulary, not ADR-0017’s controlled one.

Every field is stamped source: "manual", licence: "n/a".

The forge then shipped six work items and produced a real corpus, and nothing loaded it: seedCatalogue still called new FixtureCatalogueSource(). The status page said Forge — complete while the product ran on 333 hand-typed games.

Merging the two was measured and rejected. All 333 fixture games have no external identifier, and resolution is on identifiers only — never on title, because “Detective”, “Coup” and “Tokyo” are each several unrelated games. So every harvested twin was added rather than matched: 51 duplicate titles, Brass: Birmingham twice under two slugs and therefore two URLs.

Decision

The harvested corpus becomes the catalogue. The fixture retires from production and stays as a test corpus.

  • pnpm forge corpus writes packages/catalogue/corpus/wikidata.json, which is committed.
  • SnapshotCatalogueSource reads it, and seedCatalogue takes a source with that as the default.
  • FixtureCatalogueSource remains, and the tests pass it explicitly.

Rationale

Keeping the fixture’s mechanic tags means keeping data ADR-0017 has already ruled is not the controlled vocabulary — known-approximate values preserved for a cosmetic reason, on pages whose only justification is being trustworthy enough to rank.

The alternative that keeps both, backfilling identifiers onto the 333, is real work of a kind an agent should not do unsupervised: it is ~333 judgements about which Wikidata item a hand-typed title meant, and title matching is banned here for exactly the reason it would be needed.

Tests keep the fixture on purpose. 333 games parse in memory in milliseconds; the corpus is 3973 games and about thirty thousand provenance rows, and CI’s database is across a network — a difference that has already broken four work items in this repository. What the tests exercise is the query layer, not the size of the catalogue.

Consequences

  • Every mechanic tag on the site disappears. Wikidata carries none: across the harvest, mechanics is populated on zero records. Game pages lose their chips and WI-041-style discovery has nothing to run on until enrichment (WI-073) is approved under WI-H05. That approval is now the single highest-value thing a human can do for this product.
  • Slugs change, and slugs are URLs. catan becomes the-settlers-of-catan, because that is the game’s name in the source. Nothing links to the old ones yet, which is why this is the moment to do it.
  • The corpus is committed, at 2.7 MB. Roughly 220 KB packed. It is not reachable as a value from @tabletop/catalogue’s entry point — it is read from disk by a class exported only from @tabletop/catalogue/snapshot — because apps/web imports that package and a Worker bundle must not carry the catalogue.
  • What the fixture measured is gone. Weight is populated on no harvested record, and designers on 9%. The catalogue is broader and thinner, and the thinness is visible rather than approximated.
  • Re-harvesting is a deliberate act with a diff. The corpus changes only when somebody runs the command and commits the result, which is the same bargain as the generated client contract in ADR-0022.

Alternatives considered

Backfill identifiers onto the 333 and keep both. Best quality, and the only option that keeps the mechanic tags. Rejected as unsupervised agent work for the reason above; it remains available later, as a human-reviewed pass, and would then be a superseding ADR rather than a reversal.

Keep the fixture for tests and the corpus for production, with no shared path. Rejected: the thing under test stops being the thing deployed. What ships here is one seedCatalogue, one CatalogueSource contract, and one conformance suite that the shipped corpus itself must pass.

Wait for enrichment before loading anything. Coherent, and it would deliver a complete catalogue in one step. Rejected because it leaves the product on 333 games for as long as WI-H05 takes, and because the corpus is useful without mechanics — search, game pages and provenance all work today.