IAAILooped Models Done Right II
Towards Looped Models Done Right, Part II: Rethinking at Fixed Points
Argues that states near fixed points make the path to them matter less, enabling truncated backprop, terminal KV sharing, faster distilled prefill and cheaper RL updates. It then improves the depth prior, learned from prediction feedback with an entropy term, and introduces orthogonal input injection.
Inside the method
Simplified conceptual schematic. Consult the paper for the complete architecture.
- Recurrence family
- Core recurrence
- Depth control
- Learned depth prior during training
- KV / state strategy
- Terminal KV sharing near fixed points
Reading note
From 100M to 1.6B, the learned prior and orthogonal injection lower perplexity relative to Huginn's prior and existing injection schemes. At 1.6B, a 3× smaller KV cache matches the downstream average of fixed-depth training with a full cache.
Sources checked 2026-10-28. This catalog does not imply independent reproduction.
Related works
Cite this work
@misc{loopeddoneright22026,
title = {Towards Looped Models Done Right, Part II: Rethinking at Fixed Points},
author = {Benhao Huang and Chufan Shi and Junlin Chen and Shicheng Wen and Zhengzhong Liu and Eric Xing and Xuezhe Ma},
year = {2026},
eprint = {2610.06833},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.06833}
}