IAAIInfiLoop
Scaling to Tens of Thousands of Test-Time Iterations with Loop-Native Attention Residuals
Identifies latent overthinking, where noisy updates overwrite correct intermediate deductions, and introduces a loop-native residual connection that keeps a constant-memory running summary of past recurrent states using content-based weighting and learned temporal decay.
Inside the method
[ Recursive block ] × N + loop-native residual aggregation
Simplified conceptual schematic. Consult the paper for the complete architecture.
- Recurrence family
- Core recurrence
- Depth control
- Test-time looping up to tens of thousands of steps
- KV / state strategy
- Constant-size streaming aggregation memory over recurrent states
Reading note
A 7M-parameter model reaches 97.9% exact accuracy on Sudoku-Extreme and 13.6% pass@2 on ARC-AGI-2, and keeps improving on Sudoku-Extreme beyond 20,000 effective steps.
Sources checked 2026-10-28. This catalog does not imply independent reproduction.
Related works
Cite this work
@misc{infiloop2026,
title = {Scaling to Tens of Thousands of Test-Time Iterations with Loop-Native Attention Residuals},
author = {Pengxiang Li and Dilxat Muhtar and Di He and Guinan Su and Lu Yin and Shiwei Liu},
year = {2026},
eprint = {2610.11570},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.11570}
}