Back to r/jev

Spotting a question buried at the end of a routine status report

One of our agents sends me a dense status report every three hours: counts, ids, what it wrote. Twice this week it put a real question at the very end of one of those reports, and because the report looked routine, nobody answered it for a day or two. One of those questions was blocking real work. I took 40 of its recent reports, marked by hand which 14 actually asked for something (a decision, a confirmation, a fix routed to someone, or a reminder that an earlier question was still waiting), and asked Jev one yes/no question per report without showing it my answers. It found all 14 and wrongly flagged just one. A plain keyword search also found all 14, but it cried wolf on 18 of the 26 routine ones, because words like "pending" show up in ordinary status lines too.

What I learned

Say what "no" looks like in the instructions. I wrote "pure status (counts, known gaps being carried forward, items the agent is holding itself) is no", and that is what kept the routine reports down at 0.3 to 0.5 while the real asks sat at 0.8 to 0.96. Scores clustered well enough that 0.5 is a safe cutoff.

status: measured · questions: asks (noul) · result: 40 real reports, 14 with a buried ask: Jev at 0.5 caught 14/14 with 1 false flag (precision 0.93); a keyword search caught 14/14 but flagged 18 routine reports (precision 0.44). Every ask scored 0.63+, every status report 0.56 or less. Total cost $0.0011. · where it lives: one-task Ringer probe, jev-fyi-asks

0 comments