IAAIAttractor Models
Solve the Loop: Attractor Models for Language and Reasoning
A backbone proposes output embeddings and an attractor module refines them by solving for a fixed point, with gradients from implicit differentiation, so training memory stays constant in effective depth and iterations stop at convergence.
Inside the method
Backbone → proposal → [ Attractor module ] solved to fixed point (implicit gradients)
Simplified conceptual schematic. Consult the paper for the complete architecture.
- Recurrence family
- Adaptive depth
- Depth control
- Adaptive iterations until convergence
- KV / state strategy
- See paper
Reading note
A 770M model beats a 1.3B Transformer trained on twice the tokens; a 27M model trained on about 1000 examples reaches 91.4% on Sudoku-Extreme and 93.1% on Maze-Hard. Fixed-point training can let the solver be removed at inference.
Sources checked 2026-10-28. This catalog does not imply independent reproduction.
Related works
Cite this work
@misc{attractormodels2026,
title = {Solve the Loop: Attractor Models for Language and Reasoning},
author = {Jacob Fein-Ashley and Paria Rashidinejad},
year = {2026},
eprint = {2605.12466},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2605.12466}
}