IAAIOne Model, Two Paths
One Model, Two Paths: Balancing Single-Pass and Looped Performance in Language Models
Adapts a pretrained model so it can run either a single pass or a looped pass that reuses its middle layers. Joint weighted losses on both outputs, plus guidance from the frozen original model, stop looped training from degrading single-pass quality.
Inside the method
Pretrained stack: single pass ‖ [ Middle layers ] × 2
Simplified conceptual schematic. Consult the paper for the complete architecture.
- Recurrence family
- Core recurrence
- Depth control
- Two inference modes: single-pass or looped
- KV / state strategy
- See paper
Reading note
Most of the benefit comes from tuning the single-pass loss weight. With validation-selected weights, single-pass quality stays within a preset tolerance of ordinary continued training while looped predictions improve.
Sources checked 2026-10-28. This catalog does not imply independent reproduction.
Related works
Cite this work
@misc{onemodeltwopaths2026,
title = {One Model, Two Paths: Balancing Single-Pass and Looped Performance in Language Models},
author = {Jiarun Shen and Lei Zhao and Xiao Li},
year = {2026},
eprint = {2610.33517},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.33517}
}