IAAISCSE
Looped Transformers with Source-Centered State Evolution
Source-Centered State Evolution reconciles input conditioning with shared recurrence: inputs enter through a learned anchor and initial deviation, zero deviation maps to zero, and the anchor is an exact fixed point, removing the forcing bias of additive input injection.
Inside the method
state = anchor(x) + deviation; [ Block ] × R on deviation (zero ↦ zero)
Simplified conceptual schematic. Consult the paper for the complete architecture.
- Recurrence family
- Core recurrence
- Depth control
- Trained and extrapolated recurrent depth
- KV / state strategy
- See paper
Reading note
Improves the controlled recurrent quality frontier on WikiText-2/103, web pretraining, held-out transfer and LAMBADA; ablations credit the learned anchor and anchor invariance.
Sources checked 2026-10-28. This catalog does not imply independent reproduction.
Related works
Cite this work
@misc{scse2026,
title = {Looped Transformers with Source-Centered State Evolution},
author = {Bum Jun Kim and Kohei Hayashi and Shunsuke Kamiya and Masanori Koyama and Yusuke Iwasawa and Yutaka Matsuo},
year = {2026},
eprint = {2607.27656},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2607.27656}
}