IAAIT-LoopFormer
T-LoopFormer: Token-Level Elastic-Depth Looped Transformers with Dynamic Routing
Token-level elastic depth for looped transformers: a token-choice router lets each token pick its number of loop iterations from its hidden state, and a recursion-wise KV cache keeps a separate cache per loop for active tokens.
Inside the method
Token → router → [ Shared block ] × rᵢ (per-token depth)
Simplified conceptual schematic. Consult the paper for the complete architecture.
- Recurrence family
- Adaptive depth
- Depth control
- Token-choice routing
- KV / state strategy
- Recursion-wise KV cache per loop
Reading note
Uses LoopFormer as a baseline. Reports robust language modeling and zero-shot reasoning with the lowest decoding latency among compared models.
Sources checked 2026-10-28. This catalog does not imply independent reproduction.
Related works
Cite this work
@misc{tloopformer2026,
title = {T-LoopFormer: Token-Level Elastic-Depth Looped Transformers with Dynamic Routing},
author = {Mingqian Yu and Wenpeng Zhang and Shaobo Cui and Peilin Zhao},
year = {2026},
eprint = {2609.15160},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.15160}
}