IAAIShared Weights, Selected Computations
Shared Weights, Selected Computations: How Looped Transformers Route What Each Loop Does
Asks whether each loop of a looped transformer repeats the same operation. On graph walks, loops can advance by varying steps, and a learned linear map on the entering hidden state can steer a frozen loop toward different transitions, with attention routing as the causal pathway.
Inside the method
Hidden state → J (learned linear) → frozen [ Shared layers ] → selected transition
Simplified conceptual schematic. Consult the paper for the complete architecture.
- Recurrence family
- Core recurrence
- Depth control
- Fixed loops; analysis
- KV / state strategy
- See paper
Reading note
Changing intermediate supervision during training changes which transitions can be induced, suggesting the steering selects computations learned by the backbone rather than creating new ones.
Sources checked 2026-10-28. This catalog does not imply independent reproduction.
Related works
Cite this work
@misc{howloopsroute2026,
title = {Shared Weights, Selected Computations: How Looped Transformers Route What Each Loop Does},
author = {Jiaju Wu and Yi Hu and Muhan Zhang},
year = {2026},
eprint = {2609.39892},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.39892}
}