IAAIResidualQuant
ResidualQuant: KV Cache Quantization for Looped Transformers with 2-Bit Residuals
Exploits the high similarity of KV states across loops: it stores the final-loop KV as a reference and encodes other loops as low-precision residuals with least-squares scaling, rotations and loop-wise mixed precision.
Inside the method
KV(loop R) as reference + 2-bit residuals for loops 1…R−1
Simplified conceptual schematic. Consult the paper for the complete architecture.
- Recurrence family
- Full-stack recurrence
- Depth control
- Fixed loops of the base model
- KV / state strategy
- Final-loop KV reference plus quantized residuals (down to INT2)
Reading note
Retains accuracy close to BF16 while reducing theoretical KV storage by 80.7%; on an RTX 5090, decode throughput improves up to 2.73× at fixed batch and peak throughput up to 4.15×.
Sources checked 2026-10-28. This catalog does not imply independent reproduction.
Related works
Cite this work
@misc{residualquant2026,
title = {ResidualQuant: KV Cache Quantization for Looped Transformers with 2-Bit Residuals},
author = {Heejun Kim and Junyoung Lee and SangLyul Cho and Dongsu Han and Insu Han and Sehoon Kim},
year = {2026},
eprint = {2610.10381},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.10381}
}