IAAILoop Scaling Laws
Scaling Laws for Looped Mixture of Experts
The first scaling law that models recurrence and MoE sparsity jointly with model size and data, through a bounded, sparsity-conditional mapping from loops to effective parameters. It recovers dense and MoE laws as special cases and guides looped MoE design.
Inside the method
[ MoE block ] × R → effective parameters (sparsity-conditional)
Simplified conceptual schematic. Consult the paper for the complete architecture.
- Recurrence family
- Full-stack recurrence
- Depth control
- Law-derived recurrence
- KV / state strategy
- See paper
Reading note
Sparsity gives about 3× active-parameter efficiency and recurrence about 2× total-parameter efficiency on reasoning. At trillion-token scale and matched training compute, a looped MoE matches a roughly 2× larger non-looped MoE on reasoning benchmarks.
Sources checked 2026-10-28. This catalog does not imply independent reproduction.
Related works
Cite this work
@misc{loopscalinglawsmoe2026,
title = {Scaling Laws for Looped Mixture of Experts},
author = {Yanbei Chen and Anirudh Goyal and Raghuraman Krishnamoorthi},
year = {2026},
eprint = {2609.40316},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.40316}
}