IAAICoTFormer
CoTFormer: A Chain-of-Thought Driven Architecture with Budget-Adaptive Computation Cost at Inference
Observes that chain-of-thought resembles reapplying a model several times, except that later passes attend to earlier passes' tokens. CoTFormer mimics this at the token level and pairs it with token-wise variable depth chosen by a compute-adaptive router.
Inside the method
[ Model ] × r with attention to earlier passes' tokens; router picks rᵢ
Simplified conceptual schematic. Consult the paper for the complete architecture.
- Recurrence family
- Adaptive depth
- Depth control
- Budget-adaptive per-token depth
- KV / state strategy
- Keeps KV from every earlier pass
Reading note
Reaches accuracy close to much larger models and lowers computation through adaptive allocation. A later study (LSTM-UT) finds its full cache can hurt extrapolation on cellular-automaton tasks.
Sources checked 2026-10-28. This catalog does not imply independent reproduction.
Related works
Cite this work
@misc{cotformer2023,
title = {CoTFormer: A Chain-of-Thought Driven Architecture with Budget-Adaptive Computation Cost at Inference},
author = {Amirkeivan Mohtashami and Matteo Pagliardini and Martin Jaggi},
year = {2023},
eprint = {2310.10845},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2310.10845}
}