Debayan Mukhopadhyay, Utshab Kumar Ghosh, Shubham Chatterjee
Search systems are usually judged by a static test: give a reader model a document and a question, and see if the answer improves. That works when a document stands alone. But when a language model runs as a search agent, firing off several queries and reasoning across turns, a document can matter not for what it says but for where it sends the agent next. This paper measures that gap instead of just asserting it.
The method is clever and brutal: replay 1,000 questions with a ReAct agent, and for every document it read, delete that document and rerun the rest of the trajectory to see what breaks. Scoring documents this way turns out almost unrelated to their static usefulness score. About a third are quietly load-bearing while looking useless on their own, which they call bridge documents, and the reason is that they hand the agent a distinctive entity that reroutes the next query.
This is the abstract's account, with real numbers on HotpotQA, so read the paper for how the counterfactual scoring is defined and where it might not hold.
Retrieval systems are trained and evaluated on a static idea of usefulness: hand a document and a question to a reader model, see whether the answer improves, and score the document accordingly. The idea holds up when a document is read on its own. It breaks when a language model works as a search agent, issuing several queries and reasoning across turns, because a document can matter for what it lets the agent do next rather than for what it says about the current question.…
ArtChart: A Benchmark for Faithful Artistic Chart Generation with Integrated Text Rendering
arXiv (cs.AI) · July 17, 2026Frontier AI performance across the business disciplines: a case-grounded benchmark of knowledge work and analytical reasoning
arXiv (cs.CV) · July 17, 2026Multi-Modal Semantic Segmentation of Electrolyzer Components for Sustainable Hydrogen Technologies: A Dual-Branch Deep Learning Approach
arXiv (cs.LG) · July 17, 2026Deep and Probabilistic Models for Gene Regulatory Network Inference