IAAIBlock Recursive Transformers
Block Recursive Transformers for Structured Parameter Sharing
Argues that how shared layers are organized matters. BRT splits the stack into contiguous blocks and recurses each block before moving to the next, an intermediate topology between per-layer repetition and looping the entire stack.
Inside the method
[ Block 1 ] × r → [ Block 2 ] × r → … → [ Block n ] × r
Simplified conceptual schematic. Consult the paper for the complete architecture.
- Recurrence family
- Full-stack recurrence
- Depth control
- Static per-block recursion
- KV / state strategy
- See paper
Reading note
Improves validation NLL and downstream accuracy over static recursive sharing in decoder-only models up to 1.7B, with additional encoder-decoder and encoder-only studies; no routing modules or extra objectives.
Sources checked 2026-10-28. This catalog does not imply independent reproduction.
Related works
Cite this work
@misc{brt2026,
title = {Block Recursive Transformers for Structured Parameter Sharing},
author = {Josep Marques and Hyunyoung A. Cho and Youngeun Kim and Youjia Zhang and Aecheon Jung and Kyuhong Shim and Sungeun Hong},
year = {2026},
eprint = {2610.24998},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.24998}
}