IAAIHybrid Latent Attention
Hybrid Latent Attention for Looped Language Models
Shrinks the T-fold KV cache of looped models by keeping exact keys and values only for a sliding window of recent tokens and storing older tokens as compact latents that every loop's query reads directly. It is uptrained on frozen Ouro models, training only the added parameters.
Inside the method
Recent W tokens: exact KV ‖ older tokens: shared latent read by each loop
Simplified conceptual schematic. Consult the paper for the complete architecture.
- Recurrence family
- Full-stack recurrence
- Depth control
- Ouro T = 4 loops
- KV / state strategy
- Sliding-window exact KV plus compact per-token latents
Reading note
Cache shrinks 10.7× per token, fitting 4.0–8.8× more concurrent sequences; throughput improves 2.5× at 1K and up to 7.4× at 16K context. Retains over 97% of accuracy on math, knowledge and reasoning and 96–100% on retrieval up to 16K.
Sources checked 2026-10-28. This catalog does not imply independent reproduction.
Related works
Cite this work
@misc{hybridlatentattention2026,
title = {Hybrid Latent Attention for Looped Language Models},
author = {Yuhan Chen and Siyuan Zhang and Nan Wang and Feiyang Kang and Ruoxi Jia},
year = {2026},
eprint = {2610.07940},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.07940}
}