Noticing when a reply in someone's own Slack thread is a question for them, even with no @mention
When someone replies inside a thread the developer started, they usually don't @mention him, because it's his thread. My Slack scan only flags messages that @mention him, so a plain "any timeline for this ticket?" in his thread sat unanswered for over four hours today before I caught it. I pulled 203 real replies from his threads over the last 90 days and labeled 49 by hand: 23 ask him for something (a date, access, a file, "is this right?") and 26 don't. Ten of the 26 are tricky because they ask someone else by name in his thread. I asked Jev one yes/no question about each: is this reply asking the thread's author for something he needs to answer or do? It ranked every real ask above every non-ask. At a 0.5 cutoff it caught 22 of the 23 with no false alarms. The ten "asking someone else" replies all scored 0.15 or lower. The obvious keyword check (a question mark, "can you", "please") caught 19 and raised 5 false alarms, mostly rhetorical questions and requests aimed at other people. The one Jev missed (0.41) was "plz give [a colleague] a heads up before that starts", a request to him that's phrased as being about someone else. The whole run cost about a tenth of a cent.
What I learned
Tell Jev who started the thread and give it the opening message alongside the reply. The sentence that did the most work was "the reply may be aimed at him even though it does not @mention him". Use 0.3 to 0.5 as the cutoff, not 0.7: real asks here are short and casual ("So Aug 28?", "that's UW right?") and score lower than formal ones, and at 0.7 it missed 6.
status: measured · questions: ask (noul) · result: AUC 1.00 on 49 real replies; at 0.5 caught 22 of 23 asks with 0 false alarms out of 26; question-mark/request-phrase grep caught 19 with 5 false alarms; an @mention scan catches 0 by construction · where it lives: ~/fleet/ringer-work/jev-thread-ask