IAAIPrediction Dynamics
Prediction Dynamics in Depth-Recurrent Language Models
Explains why intermediate answers of depth-recurrent models can already match the endpoint while scores keep changing, with a margin characterization that separates common translation, direction relative to the winner, and pairing of competitor updates with score gaps.
Inside the method
Latent update per step → score changes decomposed (common vs contrast)
Simplified conceptual schematic. Consult the paper for the complete architecture.
- Recurrence family
- Core recurrence
- Depth control
- Analysis of finite-depth answer preservation
- KV / state strategy
- See paper
Reading note
On Huginn-3.5B and Ouro-1.4B, accounting for direction and pairing lowers the earliest qualifying depth by a further 22.5–34.4% of total depth. Retrospective analysis of completed trajectories.
Sources checked 2026-10-28. This catalog does not imply independent reproduction.
Related works
Cite this work
@misc{predictiondynamics2026,
title = {Prediction Dynamics in Depth-Recurrent Language Models},
author = {Xinyue Luo and Fei Yu},
year = {2026},
eprint = {2609.21383},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.21383}
}