IAAIRight Direction, Wrong Step
Right Direction, Wrong Step: Geometric Analysis of Finite-Step Failure in Looped Transformers
Separates whether an extra loop's update direction is unhelpful or its full step simply moves too far. Along the model's own update direction, many failures are locally improving directions with harmful full displacement, analyzed with a pathwise curvature decomposition.
Inside the method
state + α · update, α ∈ (0, 1] → reference utility
Simplified conceptual schematic. Consult the paper for the complete architecture.
- Recurrence family
- Core recurrence
- Depth control
- Analysis of extra iterations
- KV / state strategy
- See paper
Reading note
A fixed quarter step turns 72.2–83.2% of selected failures into gains across four settings on math and commonsense tasks, in two model families.
Sources checked 2026-10-28. This catalog does not imply independent reproduction.
Related works
Cite this work
@misc{rightdirectionwrongstep2026,
title = {Right Direction, Wrong Step: Geometric Analysis of Finite-Step Failure in Looped Transformers},
author = {Zhihao Guo and Zonghan Wu and Haizhou Du and Huan Huo and Yilei Shao and Athanasios V. Vasilakos and Qingsong Wen},
year = {2026},
eprint = {2609.16665},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.16665}
}