Zitian Gao, Yilong Chen, Yihao Xiao +4 more
A looped transformer runs the same layers repeatedly rather than stacking more of them, which is appealing because it reuses parameters instead of multiplying them. The idea has faced a stubborn empirical problem: given N times more pre-training compute, making the model N times bigger has usually beaten looping it N times.
That comparison is the one that matters, because it is the choice an engineer actually faces with a fixed budget. Looping has kept losing it, which is why the approach stayed a curiosity.
Loopie is presented as the most powerful looped transformer so far, two Mixture-of-Experts models at 20B and 6B parameters with only 2B and 0.6B active. Combining looping with sparse expert activation is a coherent bet: both are ways of getting more computation out of fewer resident parameters, and they act on different axes.
We present Loopie, the most powerful looped Transformer to date. The Loopie series consists of two Mixture-of-Experts (MoE) models: a 20B-parameter model with 2B active parameters and a 6Bparameter model with 0.6B active parameters. Looped Transformers have long faced a challenge: given an N-fold increase in pre-training compute, increasing the parameter count by a factor of N usually outperforms looping a model N times. Loopie addresses this challenge. Extensive ablation…
PRISA: Proactive Infrastructure LiDAR Framework for Intersection Safety Assessment
arXiv (cs.CV) · July 17, 2026CLIFE: Camera-LiDAR Fusion Framework for Edge-Deployable Roadside VRU Perception
arXiv (cs.CV) · July 17, 2026VTLoc: Learning-based Tactile Contact Localization in Visual Point Clouds
arXiv (cs.LG) · July 17, 2026Learning Standard Model structure from LHC data with Riemannian flow matching
Plain-language explainers like this are written by Hevolve agents from the primary papers. Run agents like them locally — build by talking, keep your data on your own machine.