IAAIClosing the Loop
Closing the Loop: Practical Training Recipes for Looped Language Models
Practical recipes for looped language models: a from-scratch pipeline that cuts Ouro's 7.7T-token budget to 310B tokens, controlled comparisons against dense models, and a minimal conversion of pretrained dense models using one learned input-mixing scalar and a smoothed exit loss.
Inside the method
Simplified conceptual schematic. Consult the paper for the complete architecture.
- Recurrence family
- Full-stack recurrence
- Depth control
- Learned exit gate
- KV / state strategy
- See paper
Reading note
The 1.4B LoopLM beats a parameter-matched dense model on all 12 benchmarks with the same data (+14 GSM8K, +10 MATH, +22 DROP) and approaches a 3.9B dense model at matched inference compute. Looped Qwen3-1.7B improves over continued dense training on every benchmark.
Sources checked 2026-10-28. This catalog does not imply independent reproduction.
Related works
Cite this work
@misc{closingtheloop2026,
title = {Closing the Loop: Practical Training Recipes for Looped Language Models},
author = {Andrei Marchenko and Viacheslav Bezrukov and Oleg Kashurin and Inessa Fedorova and Dmitry Bocharov and Yuliana Shakhvalieva and Maria Tikhonova and Valerii Ternovskii},
year = {2026},
eprint = {2610.00673},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.00673}
}