World models are the part of a reinforcement learning system that learns to imagine how an environment will unfold, but people usually judge them by how well the whole agent plays, not by whether the imagined world is any good on its own. This paper isolates that question in Atari Pong. The authors take five well-known world models, freeze them, and let a separate policy roll out inside each so they can inspect the imagined video.

All five produce visible glitches: the ball vanishes, moves wrong, or passes through the paddle. Train fresh policies entirely inside these frozen models and real-game performance drops sharply, most dramatically for DreamerV3, whose return falls from -5.5 to nearly the worst possible -21. Their proposed fix adds a reconstruction loss on important regions like the ball, and it helps in several models though not uniformly.

This comes from a truncated abstract, so read the paper for the full results.