IAAIStateLoop
StateLoop: Persistent State for Looped Transformers
Adds one persistent state per token that survives across loop iterations. A learned gate decides how much of the old state to keep and how much of the new backbone output to absorb, and the state is projected back into the stream for later iterations and the final prediction.
Inside the method
[ Backbone ] × R + gated persistent state per token
Simplified conceptual schematic. Consult the paper for the complete architecture.
- Recurrence family
- Full-stack recurrence
- Depth control
- Fixed trained loop count
- KV / state strategy
- One extra gated state vector per token; attention cache see paper
Reading note
Lowers Pile perplexity and improves the eight-task zero-shot average over a vanilla looped baseline. The perplexity advantage grows with trained loop depth.
Sources checked 2026-10-28. This catalog does not imply independent reproduction.
Related works
Cite this work
@misc{stateloop2026,
title = {StateLoop: Persistent State for Looped Transformers},
author = {Feichen Song and Boyi Zeng and He Li and Shixiang Song and Yiqin Hao and Zitong Wang and Yizhou Yang and Yifan Liu and Ziwei He and Zhouhan Lin},
year = {2026},
eprint = {2610.35544},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.35544}
}