IAAIRIPE
Prefix Keys and Values Matter Late: RIPE Caching for Self-Speculative Decoding in Looped Language Models
Shows that in recurrent-depth models the KV planes of late loops matter far more than early ones and converge geometrically. RIPE therefore stores only the deepest KV plane for most positions and keeps full depth only for a recent window, which suits self-speculative decoding with few-loop drafts and full-depth verification.
Inside the method
Simplified conceptual schematic. Consult the paper for the complete architecture.
- Recurrence family
- Core recurrence
- Depth control
- Self-speculative: draft with few loops, verify with R
- KV / state strategy
- Deepest recurrent KV plane everywhere except a recent full-depth window
Reading note
On Huginn-0125, a 64-position window removes 89% of tail-position error at 11.9% of the full cache memory (1024-token context). Measured self-speculative speed-ups are 2.09× with the full cache and 2.58× with the deepest plane.
Sources checked 2026-10-28. This catalog does not imply independent reproduction.
Related works
Cite this work
@misc{ripecaching2026,
title = {Prefix Keys and Values Matter Late: RIPE Caching for Self-Speculative Decoding in Looped Language Models},
author = {Dezhen Lu and Runguo Li and Zixiang Ni},
year = {2026},
eprint = {2610.34436},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.34436}
}