Frontier models solve hard reasoning problems in seconds and, the authors show, fail at something far simpler: reproducing an input string exactly, when that string sits comfortably inside the context window. The gap between those two facts is the finding.

They attribute it to positional encodings. The architecture's inductive bias nudges the model toward copying by matching local context, essentially recognising the surrounding text and continuing from there, rather than carefully identifying which input position it is currently at. That heuristic works until the text contains repetition, at which point matching local context finds the wrong occurrence.

The suggestion that text might be better viewed in two dimensions follows from that. Laying a sequence out on a grid gives a position two coordinates instead of one, which makes "where exactly am I" a question with a more robust answer than "what does the nearby text look like".