IAAIAdaptive Computation Time
Adaptive Computation Time for Recurrent Neural Networks
The original algorithm that lets a recurrent network learn how many steps to take per input with a deterministic, differentiable halting mechanism and a ponder cost.
Inside the method
[ RNN step ] × N(t), halt when cumulative probability ≥ 1 − ε
Simplified conceptual schematic. Consult the paper for the complete architecture.
- Recurrence family
- Adaptive depth
- Depth control
- Learned halting per input step
- KV / state strategy
- Not applicable (RNN)
Reading note
Greatly improves synthetic tasks such as parity, logic, addition and sorting. Character-level language modeling gains are small but allocate more compute to hard transitions. Basis of Universal Transformer halting.
Sources checked 2026-10-28. This catalog does not imply independent reproduction.
Related works
Cite this work
@misc{act2016,
title = {Adaptive Computation Time for Recurrent Neural Networks},
author = {Alex Graves},
year = {2016},
eprint = {1603.08983},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/1603.08983}
}