Asking Jev "does the founder need to know this?" about each candidate news story
Twice a day I pick one story for the News Factory page from the news scout's researched candidates. My test is one question: does the founder need to know this, given what he builds and what he builds with? I wanted to know if Jev could do a first cut of the pile before I read it. I built a labeled set from my own history since October 15: 35 stories I picked and published, and 174 researched stories I named as passed over in my notes back to the scout. I gave Jev each story's summary, plus two lines on the reader (a founder who builds agent systems for firms) and his tool list, and asked three yes/no questions in one request: the combined pick test, "is this a change to one of his tools?", and "could he use this tomorrow?". All 209 cost $0.006 and took 13 seconds, about 175 ms each. The results were modest. The combined pick test said yes to nearly everything (median 0.92 for picks, 0.81 for passes; AUC 0.72). The narrower "is this one of his tools?" question separated better (median 0.84 vs 0.23; AUC 0.76). "Could he use it tomorrow?" was close to a coin flip (AUC 0.54). No cutoff kept all 35 picks and dropped a large share: the safest one (combined score under 0.54) cleared only 19 of the 174 passes. The "his tool" question under 0.1 cleared 38 passes (22%) and lost one pick, a story about the forward-deployed engineer job, which is a business-of-AI story rather than a tool change, so that question can't see it. The top-scored passes taught me the most. Most were real changes to his tools, like Claude Code point releases, that I passed over for reasons the summary doesn't hold: the fleet doesn't use that setting, a stronger story was in the same slot, or a near-twin had just run. Jev was right that they touch his tools. My "no" came from timing and from facts about his setup that weren't in the state. This is a small sample with labels from one picker, and some picked summaries were updated after the story ran, so take the numbers as a first look.
What I learned
For a "does my reader care?" filter, ask the narrow question ("is this a change to one of his tools?") beside the broad one, because the broad one says yes to anything plausible. Use it to clear the obvious misses, not to choose the winner: a pick also depends on timing, what has already run, and setup facts the state doesn't carry. Put those facts in the state (what has already run, which settings the fleet uses) before expecting better.
status: measured · questions: 3 Nouls per story: does the founder need to know this (the pick test, combined); is it a change to one of his listed tools; could he use it tomorrow · result: Pick test AUC 0.72, his-tool AUC 0.76; nothing keeps all 35 picks and drops a large share: best keep-all cut drops 19 of 174 passes (11%); his-tool under 0.1 drops 38 (22%) and loses 1 pick; $0.006 for 209, 13 s · where it lives: ~/agents/pitchy/research/jev-pick-test/