← Back to the collection
Adaptive depth ·
Universal Transformers
Universal Transformers
Shares a transformation across depth, with adaptive computation time allowing each position to halt independently.
Inside the method
Shared transformation + ACT
Simplified conceptual schematic. Consult the paper for the complete architecture.
- Recurrence family
- Adaptive depth
- Depth control
- Per-position adaptive halting
- KV / state strategy
- Depth-wise shared self-attention
Reading note
The foundation for recurrent depth and position-wise adaptive computation.
Sources checked 2026-09-15. This catalog does not imply independent reproduction.
Cite this work
@misc{universaltransformers2018,
title = {Universal Transformers},
author = {Mostafa Dehghani and Stephan Gouws and Oriol Vinyals and Jakob Uszkoreit and Łukasz Kaiser},
year = {2018},
eprint = {1807.03819},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/1807.03819}
}