IAAIWhat Makes Recurrence Effective
What Makes Recurrence Effective in Looped Language Models?
Controlled experiments on when recurrence helps, where to place it, and how to condition it, across inference budgets below, within and beyond the training horizon. It proposes channel-wise history-state injection with timestep conditioning as a cheap, more robust alternative to initial-state injection.
Inside the method
Input layers → [ Recurrent block ] × r (history-state injection + timestep) → output layers
Simplified conceptual schematic. Consult the paper for the complete architecture.
- Recurrence family
- Core recurrence
- Depth control
- Variable inference budget
- KV / state strategy
- See paper
Reading note
Extra recurrence can improve reasoning beyond the training horizon while degrading knowledge. Effective depth alone does not predict behavior, and non-recurrent output layers improve robustness to under-unrolling.
Sources checked 2026-10-28. This catalog does not imply independent reproduction.
Related works
Cite this work
@misc{whatmakesrecurrenceeffective2026,
title = {What Makes Recurrence Effective in Looped Language Models?},
author = {Xinlin Zhuang and Siyuan Wang and Imran Razzak and Weiyang Liu},
year = {2026},
eprint = {2609.36636},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.36636}
}