模型配置
表中方法的准入配置位于 configs/<family>/<method>.json,并指向当前凭证集中的对应文件。下表直接从配置与凭证生成。每个输入 token 的训练 GFLOPs 相对 SSSL 锚点 4.67239832544785 计算偏差;参数量是校准的结果。
| 模型族 | 方法 | 物理块数 | FFN 宽度 | 参数量 | GFLOPs/token | 相对 SSSL | 训练阶段 ID |
|---|---|---|---|---|---|---|---|
| 基线 | dense_full |
23 | 6,144 | 1,355,810,568 | 4.674662 | +0.048% | joint |
| 循环 | deeploop |
12 | 6,144 | 742,392,722 | 4.660529 | -0.254% | joint |
| 循环 | fixed_recursive |
15 | 6,144 | 927,990,968 | 4.663054 | -0.200% | joint |
| 循环 | huginn |
14 | 6,528 | 870,581,286 | 4.649327 | -0.494% | joint |
| 循环 | ouro |
9 | 6,144 | 607,127,805 | 4.658210 | -0.304% | joint, gate_only |
| 循环 | ut_act |
12 | 6,144 | 742,394,259 | 4.660085 | -0.264% | joint |
| 稀疏 | dsa |
22 | 6,400 | 1,316,236,406 | 4.650934 | -0.459% | burnin, joint |
| 稀疏 | moba |
25 | 6,144 | 1,462,765,468 | 4.673521 | +0.024% | joint |
| 稀疏 | nsa |
23 | 6,144 | 1,381,293,832 | 4.638686 | -0.722% | joint |
| 稀疏 | qsa |
23 | 6,144 | 1,378,420,488 | 4.628037 | -0.949% | burnin, joint |
| 稀疏 | seer |
23 | 6,272 | 1,387,464,456 | 4.659564 | -0.275% | burnin, joint |
| 线性 | deltanet |
25 | 6,144 | 1,463,690,268 | 4.674979 | +0.055% | joint |
| 线性 | elu_norm |
25 | 6,144 | 1,462,768,668 | 4.651242 | -0.453% | joint |
| 线性 | elu_qknorm |
25 | 6,144 | 1,462,768,668 | 4.652243 | -0.431% | joint |
| 线性 | gdn |
23 | 6,144 | 1,411,349,680 | 4.687197 | +0.317% | joint |
| 线性 | gla |
23 | 6,144 | 1,411,207,816 | 4.642048 | -0.650% | joint |
| 线性 | kda |
23 | 6,144 | 1,412,126,344 | 4.692443 | +0.429% | joint |
| 线性 | retnet |
23 | 6,144 | 1,410,077,320 | 4.630060 | -0.906% | joint |
| 残差 | attnres |
23 | 6,144 | 1,355,882,760 | 4.679857 | +0.160% | joint |
| 残差 | hc |
23 | 6,144 | 1,356,235,700 | 4.698483 | +0.558% | joint |
| 残差 | mhc |
23 | 6,016 | 1,353,550,818 | 4.674363 | +0.042% | joint |
| 残差 | residual_standard |
23 | 6,144 | 1,355,810,568 | 4.674298 | +0.041% | joint |
| 混合 | deltanet_hybrid |
24 | 6,400 | 1,403,810,314 | 4.675326 | +0.063% | joint |
| 混合 | gdn_hybrid |
24 | 5,760 | 1,399,275,962 | 4.703800 | +0.672% | joint |
| 混合 | gdn_hybrid_swa |
22 | 6,528 | 1,329,681,662 | 4.647087 | -0.542% | joint |
| 混合 | kda_hybrid |
24 | 6,016 | 793,542,986 | 4.692309 | +0.426% | joint |
| 混合 | moba_hybrid |
25 | 6,016 | 1,452,935,068 | 4.659989 | -0.266% | joint |
| 混合 | qsa_hybrid |
24 | 5,760 | 1,405,174,202 | 4.691637 | +0.412% | burnin, joint |
| 其他 | dense (SSSL) |
24 | 6,144 | 1,384,122,122 | 4.672398 | +0.000% | joint |
循环方法跨轮复用物理块,展开后的深度会大于表内块数,详见方法来源与实现。各阶段按更新次数加权。DSA、QSA、Seer 和 QSA 混合架构先做 512 步稠密预热,Ouro 在最后 1,024 步仅训练退出门。Huginn 按固定种子逐步采样深度,凭证枚举了全部 46,875 次采样。
model 定义模型结构,protocol 给出准入协议及其指纹。calibration 保存每 token 的审计算力及偏差、方法的源码指纹、用于审计的 PyTorch 构建和凭证路径。配置由 python -m nexttransformer.protocol.calibrate 写入,再由 admit_config 检查;请通过校准流程更新。
SSSL、MoR、LoopFormer 和 Circle Loop 的历史原生配置位于 configs/native/,采用历史矩阵 FLOPs 核算。