IAAIBeNice!
BeNice! Closing Looping's Inference-Memory Gap via KV-Sharing
Makes looped models KV-shareable so earlier recurrences can reuse the final recurrence's cache. Stochastic depth during pretraining induces contractive dynamics, and the BeNice! post-training step broadcasts the final KV cache into the first recurrence while preserving quality.
Inside the method
Pretrain with stochastic depth → post-train: final-recurrence KV broadcast to earlier recurrences
Simplified conceptual schematic. Consult the paper for the complete architecture.
- Recurrence family
- Core recurrence
- Depth control
- Fixed recurrence
- KV / state strategy
- Final-recurrence KV cache shared across depth
Reading note
Matches unshared fine-tuning perplexity and composes with MLA, Kimi delta attention and fp8/mxfp8/nvfp4 KV quantization. Throughput and p90 TTFT gains come from simulated low-memory long-context serving.
Sources checked 2026-10-28. This catalog does not imply independent reproduction.
Related works
Cite this work
@misc{benice2026,
title = {BeNice! Closing Looping's Inference-Memory Gap via KV-Sharing},
author = {Hayden Prairie and Jacob M. Ryan and Taylor Berg-Kirkpatrick and Daniel Y. Fu},
year = {2026},
eprint = {2610.31633},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.31633}
}