IAAIDiffusion-Forcing Sampler
Efficient Parallel Samplers for Recurrent-Depth Models and Their Connection to Diffusion Language Models
Connects recurrent-depth models to diffusion language models and builds a diffusion-forcing sampler that decodes a new token every forward pass while earlier tokens' latents keep being refined in parallel through recurrence.
Inside the method
Each pass: new token at depth 1 ‖ earlier tokens refined further
Simplified conceptual schematic. Consult the paper for the complete architecture.
- Recurrence family
- Core recurrence
- Depth control
- Recurrence spread across positions in parallel
- KV / state strategy
- See paper
Reading note
Applies to existing 3.5B recurrent-depth transformers such as Huginn without tuning, giving up to 5× speedup; generation is argued to be strictly more expressive than autoregressive decoding at the same time budget.
Sources checked 2026-10-28. This catalog does not imply independent reproduction.
Related works
Cite this work
@misc{recurrentdepthdiffusionsampler2025,
title = {Efficient Parallel Samplers for Recurrent-Depth Models and Their Connection to Diffusion Language Models},
author = {Jonas Geiping and Xinyu Yang and Guinan Su},
year = {2025},
eprint = {2510.14961},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2510.14961}
}