IAAIreViT
One Block, Multiple Depths: Recurrent Vision Transformers with Depth-Programmed Experts
Shows a single Transformer block applied recurrently can match a full-depth vision encoder at comparable FLOPs. Depth-specific behavior comes from expressing each depth's FFN as a convex combination of a small shared expert bank, programmed by a continuous normalized-depth coordinate.
Inside the method
Simplified conceptual schematic. Consult the paper for the complete architecture.
- Recurrence family
- Full-stack recurrence
- Depth control
- Elastic depth by resampling the depth coordinate
- KV / state strategy
- Not applicable (vision encoder)
Reading note
Trained from scratch, reViT-B/16 reaches DeiT III accuracy with about 70% fewer stored parameters; a distilled 8-expert model keeps nearly all of DINOv2's linear-probe accuracy and transfers to segmentation and depth.
Sources checked 2026-10-28. This catalog does not imply independent reproduction.
Related works
Cite this work
@misc{revit2026,
title = {One Block, Multiple Depths: Recurrent Vision Transformers with Depth-Programmed Experts},
author = {Adrian Bulat and Yassine Ouali and Georgios Tzimiropoulos},
year = {2026},
eprint = {2610.12448},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.12448}
}