IAAIRetrofitted Recurrent VLMs
Retrofitting Recurrent Vision-Language Models from Pretrained Backbones
Retrofits depth recurrence into a pretrained vision-language model by pruning language-backbone layers and looping a subset of the rest, with a linear adapter merging each loop's output with earlier token representations. A three-stage recovery with distillation from the original model restores quality.
Inside the method
Simplified conceptual schematic. Consult the paper for the complete architecture.
- Recurrence family
- Core recurrence
- Depth control
- Fixed recurrence
- KV / state strategy
- See paper
Reading note
On TinyLLaVA-1.4B, without new vision-language data, it outperforms the unpruned baseline trained on the same recovery data while storing 23.3% fewer language parameters, with gains mainly on reasoning-oriented benchmarks.
Sources checked 2026-10-28. This catalog does not imply independent reproduction.
Related works
Cite this work
@misc{recurrentvlmretrofit2026,
title = {Retrofitting Recurrent Vision-Language Models from Pretrained Backbones},
author = {Madhav Iyengar and Andreas Geiger and Hilde Kühne},
year = {2026},
eprint = {2610.22075},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.22075}
}