IAAILooped Models Done Right I
Towards Looped Models Done Right, Part I: Topology, Input Injection, Recurrent-State Design
Disentangles three design axes of looped language models trained from scratch at matched parameters, logical depth and tokens: where recurrence sits, how input is injected, and how recurrent state is organized. The prelude–recurrent–coda sandwich and input injection come out ahead, favoring the Huginn-style design over Ouro.
Inside the method
Simplified conceptual schematic. Consult the paper for the complete architecture.
- Recurrence family
- Core recurrence
- Depth control
- Fixed, matched logical depth
- KV / state strategy
- See paper
Reading note
At 730M parameters and 336B tokens the full Huginn design beats Ouro on all ten benchmarks, and the advantage transfers to an 8B-A0.8B MoE trained on 500B tokens. Separate H/L states and random state initialization give mixed or negative returns.
Sources checked 2026-10-28. This catalog does not imply independent reproduction.
Related works
Cite this work
@misc{loopeddoneright12026,
title = {Towards Looped Models Done Right, Part I: Topology, Input Injection, Recurrent-State Design},
author = {Benhao Huang and Chufan Shi and Junlin Chen and Shicheng Wen and Zhengzhong Liu and Eric Xing and Xuezhe Ma},
year = {2026},
eprint = {2610.23036},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.23036}
}