IAAILoopMDM
Looped Diffusion Language Models
Shows that selectively looping the early-middle layers of masked diffusion language models improves both training efficiency and final quality, and that varying the loop count at inference gives flexible compute scaling.
Inside the method
Each denoising step: early layers → [ Early-middle layers ] × k → remaining layers
Simplified conceptual schematic. Consult the paper for the complete architecture.
- Recurrence family
- Core recurrence
- Depth control
- Loop count adjustable during sampling
- KV / state strategy
- Not an autoregressive cache setting
Reading note
Matches same-size MDMs with up to 3.3× fewer training FLOPs and gains up to 8.5 points on GSM8K; it also beats deeper non-looped MDMs at similar per-step compute. Code and weights were announced for release.
Sources checked 2026-10-28. This catalog does not imply independent reproduction.
Related works
Cite this work
@misc{loopmdm2026,
title = {Looped Diffusion Language Models},
author = {Sanghyun Lee and Chunsan Hong and Seungryong Kim and Jonghyun Lee and Jongho Park and Dongmin Park},
year = {2026},
eprint = {2605.26106},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2605.26106}
}