IAAILoop, Think & Generalize
Loop, Think, & Generalize: Implicit Reasoning in Recurrent-Depth Transformers
Studies implicit multi-hop reasoning in recurrent-depth transformers trained from scratch, covering systematic generalization over never-composed facts and depth extrapolation from up to 5-hop training to deeper compositions.
Inside the method
[ Shared layers ] × r — more iterations for more hops
Simplified conceptual schematic. Consult the paper for the complete architecture.
- Recurrence family
- Core recurrence
- Depth control
- Scaling inference-time recurrence for deeper reasoning
- KV / state strategy
- See paper
Reading note
Systematic generalization emerges through a three-stage grokking process; depth extrapolation is unlocked by more inference-time iterations, where vanilla transformers fail at both.
Sources checked 2026-10-28. This catalog does not imply independent reproduction.
Related works
Cite this work
@misc{loopthinkgeneralize2026,
title = {Loop, Think, & Generalize: Implicit Reasoning in Recurrent-Depth Transformers},
author = {Harsh Kohli and Srinivasan Parthasarathy and Huan Sun and Yuekun Yao},
year = {2026},
eprint = {2604.07822},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2604.07822}
}