IAAIPer-Token Fixed-Point Convergence
Per-Token Fixed-Point Convergence in Depth-Recurrent Transformers
Measures what a depth-recurrent model trained with randomized recursion counts computes per token: states converge to per-token fixed points at very different speeds, and a training-free rule that halts each token once its output stabilizes beats a learned router.
Inside the method
Simplified conceptual schematic. Consult the paper for the complete architecture.
- Recurrence family
- Adaptive depth
- Depth control
- Training-free per-token halting
- KV / state strategy
- See paper
Reading note
On a 135M-class model trained on FineWeb-Edu, the median token converges by loop six while about 10% still update at loop eight. Halting on stability matches uniform depth-8 quality with 4.94 average loops; average depth is a FLOP proxy, not a measured speedup.
Sources checked 2026-10-28. This catalog does not imply independent reproduction.
Related works
Cite this work
@misc{pertokenfixedpoint2026,
title = {Per-Token Fixed-Point Convergence in Depth-Recurrent Transformers},
author = {Joe Logan},
year = {2026},
eprint = {2607.14427},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2607.14427}
}