The dead row. When a benchmark refuses to move, it's a pointer, not a verdict.
Last week I shipped a feature and a board that proved it worked: every benchmark question moved except one, which sat at zero on both sides of the comparison. The temptation was to guess the fix from the board. The board can't tell you that. It took two independent bugs to make one dead row, and finding them surfaced a third bug that had shipped twenty-eight times without a symptom at all.
This is the follow-up to citation verification isn't enough. That post shipped a source-authority axis for deepdive, my local research agent, deterministic domain scoring that ranks primary sources into the limited fetch slots, so a fully-cited answer built on content farms stops looking identical to one built on the real thing. The before/after board said it worked: primary/reputable share of kept sources went 18% → 38% across the question set, committed to the repo, every question moving.
Every question except one.
| question | off | prefer | Δ prim+rep |
|-----------|------------------|-------------------|------------|
| niche-ops | 0P 1R 7U · mixed | 0P 2R 10U · mixed | +1 |
The niche-ops question, "why does nginx return 502 with an upstream sent too big header error", kept zero primary sources with authority ranking off AND on. A practitioner question with one of the best-documented answers on the internet (proxy_buffer_size, right there in the nginx docs), and the feature built to surface exactly that kind of source did nothing.
A row that refuses to move is a pointer, not a verdict. The reflex is to read the board as "the feature doesn't work on ops questions" and start guessing, expand the domain lists, tweak the ranking, add a heuristic. The board doesn't contain that information. So: evidence first.
Scoring the actual pool
The diagnostic is cheap because everything in the path is deterministic, the scorer is pure domain logic and the search adapters are plain HTTP. No LLM anywhere. Run the question's searches, score every candidate URL with the current scorer, read the table:
== general web (searxng) ==
unknown cyberciti.biz unknown getpagespeed.com
unknown bobcares.com unknown devgex.com
unknown oneuptime.com unknown merriam-webster.com
unknown davidpuplava.com unknown ambidextrous-dev.medium.com
reputable stackoverflow.com unknown copahost.com
reputable stackoverflow.com unknown andrewlock.net
== stackexchange (site=stackoverflow) ==
(zero results)
== stackexchange (site=serverfault) ==
(zero results)
Two different root causes, and neither was visible from the board.
1. The scorer was blind to the ops canon (#130). nginx.org, the literal canonical source for this question's fix, scored unknown: it's not on the curated primary list, and it carries no docs. prefix for the subdomain rule to catch. Worse, serverfault.com scored unknown too. The reputable list contains stackexchange.com, which sounds like it covers the Stack Exchange network, but the network's flagship sites live on their own domains (serverfault.com, superuser.com, askubuntu.com), not subdomains. Listing stackexchange.com never covered the sites that answer ops questions.
2. The search leg contributed nothing (#131). The question routes part of its search fan-out through the Stack Exchange API, and that API is literal-match against the question corpus. A long natural-language query returns zero items, on Stack Overflow and on Server Fault, for an error the network has answered hundreds of times. The entire candidate pool was coming from the general-web leg. Ranking can only promote what search returns; scoring can only see what it recognizes. Both halves were broken, independently.
The fix composed
Neither fix alone would have worked, which is the part worth writing down.
Fix the scorer without fixing search, and Server Fault never enters the pool, coverage of a source you never fetch is dead weight. Fix search without fixing the scorer, and Server Fault enters the pool tierless, an unknown can't win fetch slots from other unknowns.
So, two small PRs. #132 extends the curated lists, nginx.org, apache.org, kernel.org, php.net, git-scm.com as primary; the SE network's own domains as reputable. No heuristics, no denylist changes; the scorer's philosophy is boost-led and precision-first, and this is exactly the "extend as observed" path it was designed for. #133 gives the Stack Exchange adapter the same keyword ladder that fixed the identical zero-results failure for Wikipedia back in v0.23.0: when the verbatim query returns nothing, retry with 4 → 2 → 1 leading content tokens. Live before → after on the real question: 0 → 12 results on both sites.
The receipt
Same harness, same backends, after both merges (posted to the issues):
| | off | prefer |
|--------|---------------------|-------------------------|
| before | 0P 1R 7U · mixed | 0P 2R 10U · mixed |
| after | 0P 5R 7U · mixed | 1P 5R 0U · HIGH |
Even the off side improved, the ladder feeds the pool regardless of ranking. With ranking on, the fetch slots fill at 100% primary/reputable (was 42%) and the source-trust signal flips from mixed to high. Shipped as v0.27.0, and the full board re-measured at the release holds up across the question set: aggregate 23% → 44% (it was 18% → 38% before the fixes), with niche-ops, the row that started this, now the board's strongest mover, ending at 4P 5R 0U · trust: high.
Coda: the other silent empty
While cutting that release I found a second, unrelated instance of the same failure shape. The release workflow extracts each version's changelog section for the GitHub release body. Its regex ended the capture at (?=\n## \[|$), and under JavaScript's m flag, $ matches at the end of every line, so the lazy capture was allowed to stop at the first blank line it reached and grab the empty string. The regex still matched, so the "no section found" fallback never fired either.
Every release body the workflow ever produced was silently empty. Twenty-eight releases, going back to the workflow's first day in April. Nobody noticed, because an empty release body doesn't fail anything, it just quietly isn't there. The fix (#135 → #136) is a committed, unit-tested script with index slicing instead of a lazy regex, a regression test that pins the blank-line case, and backfilled bodies for every affected release.
The rule this leaves
A metric sitting at zero and an output sitting empty have the same failure mode: they don't look broken, they look like nothing. The board's dead row was two independent bugs wearing one symptom; the release notes were a bug wearing no symptom at all. In both cases what surfaced it wasn't cleverness, it was having a measurement in the first place, and treating "this number should not be zero" as a work item instead of a shrug.
That's the same stance the citation-verification post ends on, one layer up: compute the number, let it disagree with you out loud, and when it says zero, believe that it's pointing at something, just don't let it tell you what. The board found the rows; the evidence named the bugs; the re-run is the receipt. All three are public.
deepdive is the local research agent this came out of: MIT-licensed, runs on your own machine, routes through your own LLM, and reports how much to trust its own sources. The boards, the issues, and both fixes are public in the repo.
Shipping something where model output reaches a user? I do fixed-price code audits with the same stance: measurements over vibes, and a zero is a work item.