IAAIRecurTrace
RecurTrace: Adaptive Latent Reasoning with Loop-Time Memory
Improves middle-layer latent recurrence in two ways: Loop Memory Attention lets each looped layer attend to its own states from earlier iterations along the loop-time axis, and a halting head trained with oracle labels decides when extra depth still reduces loss.
Inside the method
Lower layers → [ Middle block + loop-memory attention ] × r (halting head) → upper layers
Simplified conceptual schematic. Consult the paper for the complete architecture.
- Recurrence family
- Adaptive depth
- Depth control
- Learned halting from oracle supervision
- KV / state strategy
- Each looped layer attends to its own earlier-iteration states
Reading note
On MathQA with the same looped backbone it reaches 56.9% with 2.0 loops on average, 2.2 points above the best fixed depth at matched compute, while ACT and PonderNet collapse to one loop. Gains over same-budget fine-tuning grow from 0.6B to 8B.
Sources checked 2026-10-28. This catalog does not imply independent reproduction.
Related works
Cite this work
@misc{recurtrace2026,
title = {RecurTrace: Adaptive Latent Reasoning with Loop-Time Memory},
author = {Yuxiang Wang and Kunyu Feng and Yingda Shen and Haoning Xu and Junyu Wang and Zhizheng Wu},
year = {2026},
eprint = {2609.03379},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.03379}
}