Catching stale records before they reach a decision: a pairwise "is this one out of date?" check
I build a small tool that gathers the right background before any piece of work: team notes, tickets, commits. Its worst mistake was showing an old status as if it were current. The real case: a note said an integration was BLOCKED, and a note written three hours later said access came back, but the pack put the blocked note under "already in motion" and never found the newer one. So now, for every record the pack is about to show, I pull the newest records about the same subject (same project, dated after it, never after the replay date) and ask Jev one yes/no question per pair: does the newer record show the older one no longer describes the current state? Replaying that morning's real prep, both stale records were caught (0.95 and 0.89) and the note that replaced them came into the pack. I hand-labelled every flag. At the 0.4 threshold I started with, only 7 of 15 flags were right. Every correct one scored at least 0.72, so I moved the threshold to 0.7, which kept all 7 and cut the wrong ones from 8 to 3. The whole pass costs under a cent per pack.
What I learned
Label the flags by hand before you trust a threshold; my first guess (0.4, from a different probe) was half wrong. And the remaining false flags all had one shape: a plan, a design note or a gotcha got marked "replaced" just because a newer STATUS note existed on the same project. The pairwise question has to ask whether the newer record contradicts the older record's OWN claim, not whether something newer happened nearby. That wording fix is my next test, on a second labelled case so I don't overfit one morning.
status: built · questions: 1 noul, pairwise: does the newer record show the older one no longer describes the current state? · result: replay: both known-stale records caught (0.95, 0.89); 7/7 true flags kept at 0.7; precision about 7/10; $0.007 per pack · where it lives: context_pack/supersede.py, repo commit 6ea3b87