IAAITest-Time Forecasting
Test-Time Forecasting: Recurrent Depth Enables In-Context Dynamics Prediction
Links linear attention to dynamic mode decomposition to show that looped transformers can emulate propagators of dynamical systems in context, and builds a framework in which a one-step forecaster uses gradient-descent iterations as recurrent depth.
Inside the method
Context trajectory → [ Looped block ≈ GD step ] × r → one-step forecast → rollout
Simplified conceptual schematic. Consult the paper for the complete architecture.
- Recurrence family
- Core recurrence
- Depth control
- Recurrent depth as optimization iterations
- KV / state strategy
- See paper
Reading note
Lightweight models trained on ODEs and PDEs for one-step prediction generalize to out-of-distribution dynamics during autoregressive rollout.
Sources checked 2026-10-28. This catalog does not imply independent reproduction.
Related works
Cite this work
@misc{testtimeforecasting2026,
title = {Test-Time Forecasting: Recurrent Depth Enables In-Context Dynamics Prediction},
author = {Jeffrey B. Lai and John Quinn and Anthony Bao and William Gilpin},
year = {2026},
eprint = {2610.19552},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.19552}
}