IAAILoOPSD
LoOPSD: On-Policy Self-Distillation with Looped Teachers
Builds a stronger teacher from the same frozen weights by searching which layers to repeat, then distills that looped teacher's predictions on student-generated prefixes back into the student, alternating search and distillation over several rounds.
Inside the method
Search repeated layers → looped teacher → on-policy distillation → repeat
Simplified conceptual schematic. Consult the paper for the complete architecture.
- Recurrence family
- Core recurrence
- Depth control
- Teacher repeats searched layers; student runs a single pass
- KV / state strategy
- See paper
Reading note
On GSM8K with a 400-token thinking budget, Qwen3-1.7B rises from 42.30% to 79.61%, with further gains in later rounds. Evaluated on mathematical and logical reasoning tasks.
Sources checked 2026-10-28. This catalog does not imply independent reproduction.
Related works
Cite this work
@misc{loopsd2026,
title = {LoOPSD: On-Policy Self-Distillation with Looped Teachers},
author = {Wei Fan and Qingyue Yang and Chao-Hong Tan and Qian Chen and Wen Wang and Xiangang Li and Kejiang Chen and Weiming Zhang and Nenghai Yu},
year = {2026},
eprint = {2610.18162},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.18162}
}