IAAILooped Transformers as Programmable Computers
Looped Transformers as Programmable Computers
Programs a fixed-weight transformer and places it in a loop so the input sequence acts as a punchcard of instructions and memory, emulating a small instruction-set computer with a 13-layer looped encoder.
Inside the method
Punchcard input → [ 13-layer programmed transformer ] × steps
Simplified conceptual schematic. Consult the paper for the complete architecture.
- Recurrence family
- Full-stack recurrence
- Depth control
- Loop until the program halts
- KV / state strategy
- Not applicable (constructive encoder)
Reading note
A constructive theory result: the looped transformer emulates a calculator, a basic linear-algebra library and in-context learning via backpropagation. Weights are hand-constructed, not trained.
Sources checked 2026-10-28. This catalog does not imply independent reproduction.
Related works
Cite this work
@misc{programmablecomputers2023,
title = {Looped Transformers as Programmable Computers},
author = {Angeliki Giannou and Shashank Rajput and Jy-yong Sohn and Kangwook Lee and Jason D. Lee and Dimitris Papailiopoulos},
year = {2023},
eprint = {2301.13196},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2301.13196}
}