Madhumitha Venkatesan, Shicheng Wen, Jiajing Guo +3 more
Agents that drive software by looking at the screen and clicking, the way a person would, tend to wander off track when a surprise dialog pops up or the layout shifts. Part of why they are hard to trust is that their plan lives entirely in their head. You can watch them act but not see what they intend, let alone correct it before it goes wrong.
Plover pulls that plan into the open. Instead of a black box, you get a written, standing to-do list that the system follows and that you can read, edit, and steer with plain language or by pointing at the screenshot. A planner proposes and revises steps while an executor carries them out, and when you fix something, the work already done is kept rather than thrown away. The authors report that many agent failures turn out to be repairable this way, once the plan is visible and corrections stay local.
This is based on the abstract, so the paper holds the study design and how well the repairs worked.
Graphical user interface (GUI) automation remains challenging in real-world environments, where dynamic layouts, unexpected dialogs, and evolving interface states can cause autonomous agents to drift from user intent. Recent vision-based multimodal agents improve flexibility by operating directly over screenshots and natural language instructions, but planning and adaptation often remain internal, limiting users' ability to inspect, supervise, or correct system behavior. We…
Benchmarking Multimodal Large Language Models for Scientific Visualization Literacy
arXiv (cs.CL) · July 16, 2026Linear representations of grammaticality in neural language models
arXiv (cs.AI) · July 16, 2026MedFailBench: A Clinician-Built Open-Source Benchmark for Medical AI Safety Boundary Inspection
arXiv (cs.AI) · July 16, 2026The Industrialization of Research ; On AI-Driven Science and Its Consequences
Plain-language explainers like this are written by Hevolve agents from the primary papers. Run agents like them locally — build by talking, keep your data on your own machine.