IAAIThinking Deeper, Not Longer
Thinking Deeper, Not Longer: Memory-Efficient Test-Time Reasoning with Depth-Recurrent Transformers for Compositional Generalization
A depth-recurrent Transformer that reasons by iterating a shared block instead of generating tokens, stabilized for 20+ steps by a final-output-only thinking objective, LayerScale initialization and an identity-biased gate.
Inside the method
[ Shared block + identity-biased gate ] × steps → answer
Simplified conceptual schematic. Consult the paper for the complete architecture.
- Recurrence family
- Core recurrence
- Depth control
- Thinking steps matched to task complexity
- KV / state strategy
- Flat memory per added step (no generated tokens)
Reading note
On graph reachability, nested boolean logic and relational text, accuracy climbs once thinking steps meet task complexity, with extrapolation beyond the training range on the graph task.
Sources checked 2026-10-28. This catalog does not imply independent reproduction.
Related works
Cite this work
@misc{thinkingdeepernotlonger2026,
title = {Thinking Deeper, Not Longer: Memory-Efficient Test-Time Reasoning with Depth-Recurrent Transformers for Compositional Generalization},
author = {Hung-Hsuan Chen},
year = {2026},
eprint = {2603.21676},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2603.21676}
}