IAAILoopCD
Decoding Looped Transformers Better for (Almost) Free
Training-free contrastive decoding that contrasts a looped model's final prediction with an earlier recurrent pass, which serves as a built-in weak model. It works in logit space with one extra output pass or in hidden-state space with no extra output cost.
Inside the method
Final-loop prediction − λ · earlier-loop prediction → token
Simplified conceptual schematic. Consult the paper for the complete architecture.
- Recurrence family
- Full-stack recurrence
- Depth control
- Full depth, or about half the loops with guidance
- KV / state strategy
- See paper
Reading note
Raises Ouro-2.6B-Thinking's AIME 2024 pass@1 from 61.88% to 73.33% and Huginn's HumanEval pass@1 from 22.56% to 31.71%. With guidance, halving the loops matches or beats unguided full depth, saving 22.5–48.2% forward FLOPs.
Sources checked 2026-10-28. This catalog does not imply independent reproduction.
Related works
Cite this work
@misc{loopcd2026,
title = {Decoding Looped Transformers Better for (Almost) Free},
author = {Weihao Liu and Huangjie Zheng and Tianrong Chen and Rohit Dilip and Richard He Bai and Yizhu Jiao and Yuyang Wang and Ruixiang Zhang},
year = {2026},
eprint = {2610.02185},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.02185}
}