Frontier architectures, one framework
Compare dense, sparse, linear, looped, residual and hybrid architectures in one codebase. A new method is one plugin folder.
NextTransformer
Train frontier Transformer architectures under one explicit protocol, compare them fairly, and let AI agents research the next one.
Why NextTransformer
Section titled “Why NextTransformer”A fair, explicit protocol
Same data, tokens, optimizer, schedule and evaluation for every model, with training compute matched to ±1% by an operator-level ledger and audited over the whole run.
49B tokens of pretraining
Train a ~1.4B-parameter model on 49B tokens on one 8-GPU node in 2–4 days. Recipes for 400B tokens and larger models arrive in November 2026.
Agent-native research
Hand the arena task to your agent and let it research the next architecture. The full agent-native version ships by November 14, 2026.
Start here
Section titled “Start here”Quick startInstall, prepare data, and train on one node or many.
ProtocolWhat is fixed, what is matched, and how runs are audited.
Add a methodWrite a plugin, calibrate it, and get it admitted.
LeaderboardEvery finished run against the Dense Baseline.