IAAIAdaptive Loops and Memory
Adaptive Loops and Memory in Transformers: Think Harder or Know More?
Combines adaptive per-layer looping, where each block learns to iterate with a halting mechanism, with gated memory banks that add learned storage, to test whether models should think harder or know more.
Inside the method
Layer ℓ: [ Block ] × rℓ (halting) + gated memory bank
Simplified conceptual schematic. Consult the paper for the complete architecture.
- Recurrence family
- Adaptive depth
- Depth control
- Per-layer learned halting
- KV / state strategy
- Gated memory banks
Reading note
Looping mainly helps math while memory recovers commonsense performance; together they beat an iso-FLOP baseline with three times as many layers on math. Early layers loop little and later layers loop and read memory more.
Sources checked 2026-10-28. This catalog does not imply independent reproduction.
Related works
Cite this work
@misc{adaptiveloopsmemory2026,
title = {Adaptive Loops and Memory in Transformers: Think Harder or Know More?},
author = {Markus Frey and Behzad Shomali and Ali Hamza Bashir and David Berghaus and Joachim Koehler and Mehdi Ali},
year = {2026},
eprint = {2603.08391},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2603.08391}
}