IAAIMELT
Memory-Efficient Looped Transformer: Decoupling Compute from Memory in Looped Language Models
The Memory-Efficient Looped Transformer keeps one KV cache per layer shared across reasoning loops and updated by a learnable gate, decoupling reasoning depth from memory. It is obtained from Ouro by chunk-wise two-phase post-training with interpolation and attention-aligned distillation.
Inside the method
[ Layer ] × R with one gated KV cache per layer across loops
Simplified conceptual schematic. Consult the paper for the complete architecture.
- Recurrence family
- Full-stack recurrence
- Depth control
- Fixed reasoning loops
- KV / state strategy
- Single gated KV cache per layer shared across loops
Reading note
MELT models fine-tuned from Ouro outperform standard LLMs of similar size at a comparable memory footprint, far below Ouro's.
Sources checked 2026-10-28. This catalog does not imply independent reproduction.
Related works
Cite this work
@misc{melt2026,
title = {Memory-Efficient Looped Transformer: Decoupling Compute from Memory in Looped Language Models},
author = {Victor Conchello Vendrell and Arnau Padres Masdemont and Niccolò Grillo and Jordi Ros-Giralt and Arash Behboodi and Fabio Valerio Massoli},
year = {2026},
eprint = {2605.07721},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2605.07721}
}