IAAILatent Recurrent Search
Not All Thinking is Created Equal: Latent Reasoning Discovers a Recurrent Search Algorithm for Depth Generalization
Compares vanilla, chain-of-thought, pause-token and two latent-reasoning variants of one backbone trained on extended multi-hop reasoning. Only the latent variants generalize to longer hops, and circuit analysis finds a sparse recurrent forward-search circuit propagating reachability across steps.
Inside the method
Retrieve relations (attention) → update reachability (MLP + residual) × steps → match
Simplified conceptual schematic. Consult the paper for the complete architecture.
- Recurrence family
- Core recurrence
- Depth control
- Recurrent latent steps beyond the training depth
- KV / state strategy
- See paper
Reading note
Strong in-distribution accuracy does not imply depth generalization: CoT and pause-token models rely on local graph features. Results are on the synthetic ProsQA-Ext task.
Sources checked 2026-10-28. This catalog does not imply independent reproduction.
Related works
Cite this work
@misc{latentrecurrentsearch2026,
title = {Not All Thinking is Created Equal: Latent Reasoning Discovers a Recurrent Search Algorithm for Depth Generalization},
author = {Huzi Cheng and Zhewei Zhang},
year = {2026},
eprint = {2609.35643},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.35643}
}