IAAIRecursive CLIP
Recursive CLIP: Vision-Language Distillation with Looped Transformer Blocks
Asks how much block reuse CLIP tolerates. Recursive backbones apply N blocks M times with N·M fixed at ViT-B depth, either in the vision encoder only or in both encoders, trained contrastively or by distillation from a strong ViT-B/16 teacher.
Inside the method
[ N blocks ] × M (N · M = ViT-B depth), vision and optionally text encoder
Simplified conceptual schematic. Consult the paper for the complete architecture.
- Recurrence family
- Full-stack recurrence
- Depth control
- Fixed recursion
- KV / state strategy
- Not applicable (encoders)
Reading note
CLIP tolerates substantial recurrent reuse, though the effect depends on encoder and task. Evaluated on zero-shot classification, retrieval, few-shot learning, grounding and concept unlearning with CC3M and CC12M training.
Sources checked 2026-10-28. This catalog does not imply independent reproduction.
Related works
Cite this work
@misc{recursiveclip2026,
title = {Recursive CLIP: Vision-Language Distillation with Looped Transformer Blocks},
author = {Szymon Mazurek and Alberto Presta and Michał Byra},
year = {2026},
eprint = {2610.34563},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.34563}
}