IAAIElastic Latent Recurrence
One Model to Loop Them All: Elastic Latent Recurrence for Language Models
A training recipe that removes the one-loop tax of looped language models, so one checkpoint at one loop matches a non-recurrent model of the same size and gains accuracy with each additional loop. It uses gated state updates with learned step size, mixed loop counts per update, and loop-index and budget conditioning.
Inside the method
Simplified conceptual schematic. Consult the paper for the complete architecture.
- Recurrence family
- Adaptive depth
- Depth control
- Elastic: 1 to several loops from one checkpoint
- KV / state strategy
- Keeping only the last loop's KV holds memory at the one-loop level
Reading note
On 4B dense and 30B-A3B MoE hybrids, one loop is within about a point of matched baselines and four loops add 7.25 and 4.75 points on a ten-task average. Post-training yields Koi-4B. Mixed depths can be introduced late in pretraining.
Sources checked 2026-10-28. This catalog does not imply independent reproduction.
Related works
Cite this work
@misc{elasticlatentrecurrence2026,
title = {One Model to Loop Them All: Elastic Latent Recurrence for Language Models},
author = {Faisal Ladhak and Shantanu Acharya and Santiago Akle Serano and Simeng Sun and Boris Ginsburg and Krishna C Puvvada},
year = {2026},
eprint = {2610.12582},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.12582}
}