weekly status

Search relaunch, week 6

Indexing is done and dark-launched; relevance is the open question. The shadow comparison puts the new engine ahead on head queries and behind on long-tail, which is the expected trade at this stage.

118 msp95 latency-64 0.71ndcg@10, head+0.05 0.58ndcg@10, tail-0.03 99.4%index coverage

Milestones

Reindex on the new schema
weeks 1-4
Full corpus, verified against the old index by document count and spot checks. Shadow traffic comparison
weeks 5-7relevance
Every production query runs on both engines; judgments collected on the diffs. Long-tail recovery
week 7relevancedata
Blocked on the synonym table export from the old engine — owner found, ETA Tuesday. Cutover behind a flag
week 9
5% of traffic, then doubling daily if guardrails hold.

The long-tail gap

The old engine's synonym table carries fifteen years of manual curation; the new engine currently ships without it. On tail queries where a synonym was the match, we lose the document entirely, which is most of the −0.03.

Evidence: 61% of tail regressions disappear with the table patched in

A one-hour experiment loading the March export into the new engine's synonym slot recovered 214 of 349 regressed queries. The remainder split between stemming differences and genuine ranking changes.

Next week

  1. Import the synonym table export when it lands Tuesday.
  2. Re-run the shadow comparison on the tail set.
  3. Draft the cutover guardrails (error rate, zero-result rate, p95).