IAAILooped UDLM
Looped Uniform Diffusion Language Models
Applies selective looping, which repeats a few early layers inside each denoising step, to uniform diffusion language models without changing their objective or sampler.
Inside the method
Each denoising step: [ Early layers ] × k → remaining layers
Simplified conceptual schematic. Consult the paper for the complete architecture.
- Recurrence family
- Core recurrence
- Depth control
- Loop count sampled during training; chosen at inference
- KV / state strategy
- Not an autoregressive cache setting; see paper
Reading note
Reaches a non-looped UDLM's validation NLL with up to 2.8× (OpenWebText) and 3.3× (FineWeb-Edu) fewer training FLOPs. Sampling small loop counts more often during training uses 12% fewer loops downstream without hurting accuracy.
Sources checked 2026-10-28. This catalog does not imply independent reproduction.
Related works
Cite this work
@misc{loopedudlm2026,
title = {Looped Uniform Diffusion Language Models},
author = {Saiteja Utpala and Chieh-Hsin Lai and Yuki Mitsufuji and Praneeth Vepakomma},
year = {2026},
eprint = {2610.30066},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.30066}
}