IAAIFully Looped Transformer
Simply Stabilizing the Loop via Fully Looped Transformer
Traces training instability of deep looping to gradient oscillation and residual explosion, and fixes them with two parameter-free changes: distributing inter-loop signals across all layers, and attention injection that reuses the existing attention block.
Inside the method
[ All layers, inter-loop signal at every layer ] × R + attention injection
Simplified conceptual schematic. Consult the paper for the complete architecture.
- Recurrence family
- Full-stack recurrence
- Depth control
- Stable up to 12 loops; adjustable at inference
- KV / state strategy
- See paper
Reading note
Trains stably to 12 iterations where baseline looped models collapse, and improves average downstream performance by up to 13.2% in milder settings.
Sources checked 2026-10-28. This catalog does not imply independent reproduction.
Related works
Cite this work
@misc{fullyloopedtransformer2026,
title = {Simply Stabilizing the Loop via Fully Looped Transformer},
author = {Rao Fu and Zixuan Yang and Jiankun Zhang and Jing Ma and Hechang Chen and Yu Li and Yi Chang},
year = {2026},
eprint = {2605.18797},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2605.18797}
}