IAAIMeSH
MeSH: Memory-as-State-Highways for Recursive Transformers
Memory-as-State-Highways externalizes state management of recursive transformers into an explicit memory buffer with lightweight routers, addressing undifferentiated computation across iterations and overload of a single hidden state.
Inside the method
[ Recursive core ] × r ↔ memory buffer (read / write routers)
Simplified conceptual schematic. Consult the paper for the complete architecture.
- Recurrence family
- Core recurrence
- Depth control
- Fixed iterations
- KV / state strategy
- Explicit memory buffer across iterations
Reading note
On Pythia 160M–6.9B it beats recursive baselines and, at 1.4B, outperforms the larger non-recursive model by 1.06 points with 33% fewer non-embedding parameters.
Sources checked 2026-10-28. This catalog does not imply independent reproduction.
Related works
Cite this work
@misc{mesh2025,
title = {MeSH: Memory-as-State-Highways for Recursive Transformers},
author = {Chengting Yu and Xiaobo Shu and Yadao Wang and Yizhen Zhang and Haoyi Wu and Jiaang Li and Rujiao Long and Ziheng Chen and Yuchi Xu and Wenbo Su and Bo Zheng},
year = {2025},
eprint = {2510.07739},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2510.07739}
}