IAAIDepth-Asynchronous Self-Speculation
Shallow Queries, Mature Values: Depth-Asynchronous Self-Speculation for Looped Transformers
Observes that queries and keys reach their final-depth form earlier than values. DAS lets shallow draft queries read full-depth prefix values without extra recurrence, and DAS-Wave adds parallel refinement, progressive block growth and an independent full-depth verifier.
Inside the method
Shallow draft queries → mature full-depth prefix values → full-depth verifier
Simplified conceptual schematic. Consult the paper for the complete architecture.
- Recurrence family
- Core recurrence
- Depth control
- Draft at early depth, verify at full depth
- KV / state strategy
- Draft reads full-depth prefix values (Mature-V)
Reading note
Across four recurrent-model checkpoints on math and code, DAS-Wave reaches 4.00–6.96× mean throughput speedup over full-depth autoregressive decoding in the same stack.
Sources checked 2026-10-28. This catalog does not imply independent reproduction.
Related works
Cite this work
@misc{dasselfspeculation2026,
title = {Shallow Queries, Mature Values: Depth-Asynchronous Self-Speculation for Looped Transformers},
author = {Guanghao Li and Zihan Su and Hao Yu and Jinyang Jiang and Tao Ren and Zehao Li and Feng Lu and Ming Tang and Chun Yuan},
year = {2026},
eprint = {2609.34538},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.34538}
}