Real-world bug reports are full of pictures: a screenshot of a mangled layout, an error dialog, a log dump. But the task of pinpointing which file or function is responsible has mostly been tested as if reports were plain text. Where multimodal benchmarks exist, they judge the whole repair job at once, muddying whether the image helped find the bug or just helped write the fix.

MM-IssueLoc separates those questions. It is a benchmark of 652 real issue-and-fix pairs across 23 programming languages, with images tagged by type and relevance, grading localization down to the file and function level, once with the image and once without. The honest finding: current systems are not good at this yet, with the best agent landing the right file in its top five only about 39% of the time. And strong scores on text-heavy coding benchmarks do not carry over cleanly here.

This recap is based on the abstract, so the paper has the full protocol and scoreboard.