← Back to the collection
Full-stack recurrence ·
Relaxed Recursive Transformers
Relaxed Recursive Transformers: Effective Parameter Sharing with Layer-wise LoRA
Converts pretrained Transformers into recursive models, relaxing sharing with depth-specific low-rank adaptations.
Inside the method
[ Shared layers + depth LoRA ] × R
Simplified conceptual schematic. Consult the paper for the complete architecture.
- Recurrence family
- Full-stack recurrence
- Depth control
- Recursion budget
- KV / state strategy
- Continuous depth-wise batching
Reading note
The reported batching throughput gains are based on analysis and simulation; distinguish them from end-to-end serving measurements.
Sources checked 2026-09-15. This catalog does not imply independent reproduction.
Cite this work
@misc{relaxedrecursive2024,
title = {Relaxed Recursive Transformers: Effective Parameter Sharing with Layer-wise LoRA},
author = {Sangmin Bae and Adam Fisch and Hrayr Harutyunyan and Ziwei Ji and Seungyeon Kim and Tal Schuster},
year = {2024},
eprint = {2410.20672},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2410.20672}
}