IAAIReasoning with Latent Thoughts
Reasoning with Latent Thoughts: On the Power of Looped Transformers
Argues that many reasoning problems need depth rather than parameters: a k-layer transformer looped L times nearly matches a kL-layer model on synthetic reasoning, backed by theory that iterative algorithms map onto looped models.
Inside the method
[ k layers ] × L ≈ kL-layer model
Simplified conceptual schematic. Consult the paper for the complete architecture.
- Recurrence family
- Full-stack recurrence
- Depth control
- Fixed loops L
- KV / state strategy
- See paper
Reading note
The benefit carries to language modeling: looped models compete with or beat deeper non-looped models on downstream reasoning, and scaling tracks effective depth. Also connects looping to chain-of-thought.
Sources checked 2026-10-28. This catalog does not imply independent reproduction.
Related works
Cite this work
@misc{latentthoughtslooped2025,
title = {Reasoning with Latent Thoughts: On the Power of Looped Transformers},
author = {Nikunj Saunshi and Nishanth Dikkala and Zhiyuan Li and Sanjiv Kumar and Sashank J. Reddi},
year = {2025},
eprint = {2502.17416},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2502.17416}
}