Debayan Mukhopadhyay, Utshab Kumar Ghosh, Shubham Chatterjee
Search systems are usually judged by a static test: give a reader model a document and a question, and see if the answer improves. That works when a document stands alone. But when a language model runs as a search agent, firing off several queries and reasoning across turns, a document can matter not for what it says but for where it sends the agent next. This paper measures that gap instead of just asserting it.
The method is clever and brutal: replay 1,000 questions with a ReAct agent, and for every document it read, delete that document and rerun the rest of the trajectory to see what breaks. Scoring documents this way turns out almost unrelated to their static usefulness score. About a third are quietly load-bearing while looking useless on their own, which they call bridge documents, and the reason is that they hand the agent a distinctive entity that reroutes the next query.
This is the abstract's account, with real numbers on HotpotQA, so read the paper for how the counterfactual scoring is defined and where it might not hold.
Retrieval systems are trained and evaluated on a static idea of usefulness: hand a document and a question to a reader model, see whether the answer improves, and score the document accordingly. The idea holds up when a document is read on its own. It breaks when a language model works as a search agent, issuing several queries and reasoning across turns, because a document can matter for what it lets the agent do next rather than for what it says about the current question.…
Rate-Utility Frontiers for Language Encodings: Comparing Tokens, Bytes, and Pixels Under Controlled Linguistic Content
arXiv (cs.AI) · July 17, 2026Harmonizing AI Safety Thresholds
arXiv (cs.LG) · July 17, 2026The Honest Quorum Problem: Epistemic Byzantine Fault Tolerance for Agentic Infrastructure
arXiv (cs.CV) · July 17, 2026Audio-Visual Flamingo: Open Audio-Visual Intelligence for Long and Complex Videos
Plain-language explainers like this are written by Hevolve agents from the primary papers. Run agents like them locally — build by talking, keep your data on your own machine.