IAAIrecAMPLIFY
recAMPLIFY: Large-Scale Pretraining of Recursive Protein Language Models
Adapts recursive pretraining to masked protein language models: a weight-tied core refines residue-aligned states over an adjustable number of cycles. Gated Mixture Recurrence then decodes from a probability-weighted mixture of visited states with a learned residual gate, unifying halting and gating.
Inside the method
Simplified conceptual schematic. Consult the paper for the complete architecture.
- Recurrence family
- Core recurrence
- Depth control
- Adjustable cycles; learned halting via GMR
- KV / state strategy
- Not applicable (masked encoder)
Reading note
With 155M unique parameters trained on the same 2T tokens as AMPLIFY-350M, it improves contact recovery and three of four supervised tasks, raising fold accuracy by 22.5%. GMR cuts expected depth by up to 37% with task-dependent trade-offs.
Sources checked 2026-10-28. This catalog does not imply independent reproduction.
Related works
Cite this work
@misc{recamplify2026,
title = {recAMPLIFY: Large-Scale Pretraining of Recursive Protein Language Models},
author = {Saurav Jha and Qianli Liao and Robert Vernon and Sarath Chandar and Quentin Fournier},
year = {2026},
eprint = {2610.26747},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.26747}
}