IAAIMixerLoop
Allocating Recurrent Compute in Looped Language Models
Asks what should loop. Using Iterative Transport Rank to measure the new cross-position influence each application adds, it repeats each Gated DeltaNet mixer while applying the dense FFN only once.
Inside the method
[ Mixer × r → FFN × 1 ] per layer
Simplified conceptual schematic. Consult the paper for the complete architecture.
- Recurrence family
- Core recurrence
- Depth control
- Mixer-only recurrence
- KV / state strategy
- Recurrent state of Gated DeltaNet; see paper
Reading note
At 15M, MixerLoop beats full-block looping on aggregate CORE; at 110M it keeps 41.5% of the CORE improvement while cutting recurrent-backbone projection FLOPs by 45.9%.
Sources checked 2026-10-28. This catalog does not imply independent reproduction.
Related works
Cite this work
@misc{mixerloop2026,
title = {Allocating Recurrent Compute in Looped Language Models},
author = {Ruhai Lin and Yiyang Guo and Rui-Jie Zhu and Hao Ye and Jason K. Eshraghian},
year = {2026},
eprint = {2608.18230},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2608.18230}
}