IAAIMoR-MLLM
MoR-MLLM: Mixture of Recursions for Efficient Multimodal Large Language Models
Applies Mixture-of-Recursions to multimodal LLMs so each token picks its recursion depth, spending more on visually or linguistically hard tokens. A three-stage tuning schedule and entropy-regularized routing loss stabilize recursive sparsity.
Inside the method
Visual + text tokens → router → [ Shared block ] × rᵢ
Simplified conceptual schematic. Consult the paper for the complete architecture.
- Recurrence family
- Adaptive depth
- Depth control
- Per-token recursion depth
- KV / state strategy
- See paper
Reading note
Reduces training memory and computation relative to recent tiny MLLMs while retaining strong performance on vision-language tasks.
Sources checked 2026-10-28. This catalog does not imply independent reproduction.
Related works
Cite this work
@misc{mormllm2026,
title = {MoR-MLLM: Mixture of Recursions for Efficient Multimodal Large Language Models},
author = {Pengcheng Zheng and Chaoning Zhang and Jiaxin Yan and Sihan Cao and Jianwei Zhang and Xudong Wang and Jiaquan Zhang and Jewon Lee and Tae-Ho Kim and Yang Yang and Heng Tao Shen},
year = {2026},
eprint = {2610.08830},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.08830}
}