IAAIEncode–Think–Decode
Encode, Think, Decode: Scaling test-time reasoning with recursive latent thoughts
Mid-trains a base model to iterate over a small subset of reasoning-relevant layers, chosen from interpretability evidence, while keeping architecture, parameter count, hyperparameters and data unchanged.
Inside the method
Encode layers → [ Think layers ] × k → Decode layers
Simplified conceptual schematic. Consult the paper for the complete architecture.
- Recurrence family
- Core recurrence
- Depth control
- Fixed iterations; adaptive per-token depth explored
- KV / state strategy
- See paper
Reading note
On OLMo-2 1B Base, iterating the selected layers gives +28.4% relative on GSM8K and +36% on MATH, with gains across 17 reasoning benchmarks.
Sources checked 2026-10-28. This catalog does not imply independent reproduction.
Related works
Cite this work
@misc{etd2025,
title = {Encode, Think, Decode: Scaling test-time reasoning with recursive latent thoughts},
author = {Yeskendir Koishekenov and Aldo Lipani and Nicola Cancedda},
year = {2025},
eprint = {2510.07358},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2510.07358}
}