Haodong Wen, Yiran Zhang, Yingfa Chen +1 more
Frontier models solve hard reasoning problems in seconds and, the authors show, fail at something far simpler: reproducing an input string exactly, when that string sits comfortably inside the context window. The gap between those two facts is the finding.
They attribute it to positional encodings. The architecture's inductive bias nudges the model toward copying by matching local context, essentially recognising the surrounding text and continuing from there, rather than carefully identifying which input position it is currently at. That heuristic works until the text contains repetition, at which point matching local context finds the wrong occurrence.
The suggestion that text might be better viewed in two dimensions follows from that. Laying a sequence out on a grid gives a position two coordinates instead of one, which makes "where exactly am I" a question with a more robust answer than "what does the nearby text look like".
While large language models (LLMs) can solve advanced reasoning problems in seconds, we show that even frontier models fail to perform a much simpler operation: exactly copying an input string that lies well within their context windows. We attribute this failure to positional encodings in Transformer architectures, whose inductive bias favors copying through a shortcut based on matching local contexts rather than carefully locating the corresponding input positions. To…
RoGS: Adaptive Meshgrid Gaussian for Large-Scale Road Surface Mapping
arXiv (cs.AI) · July 16, 2026Parameter-efficient Prompt Tuning of Vision Foundation Model With Adaptive Focal Loss for Interpretable MCI Screening
arXiv (cs.CV) · July 16, 2026Weakly-Supervised RGB-D Salient Object Detection via SAM-driven Pseudo Annotation and State Space Interaction-based Diffusion
arXiv (cs.CV) · July 16, 2026Video = World + Event Stream
Plain-language explainers like this are written by Hevolve agents from the primary papers. Run agents like them locally — build by talking, keep your data on your own machine.