IAAIParallel Loop Transformer
Parallel Loop Transformer for Efficient Test-Time Computation Scaling
Removes the sequential latency of looping with Cross-Loop Parallelism, which computes different loops for different tokens in the same pass, and controls memory by sharing the first loop's KV cache with later loops combined through gated sliding-window attention.
Inside the method
Token t: loop 1 ‖ token t−1: loop 2 ‖ … in one pass; shared KV + gated SWA
Simplified conceptual schematic. Consult the paper for the complete architecture.
- Recurrence family
- Full-stack recurrence
- Depth control
- Fixed loops executed in parallel across tokens
- KV / state strategy
- First-loop KV shared across loops plus gated sliding-window attention
Reading note
Reports looped-model accuracy with almost no extra latency or memory over a standard transformer. LoopCoder-v2 later trains 7B PLT coders and finds two loops optimal.
Sources checked 2026-10-28. This catalog does not imply independent reproduction.
Related works
Cite this work
@misc{plt2025,
title = {Parallel Loop Transformer for Efficient Test-Time Computation Scaling},
author = {Bohong Wu and Mengzhao Chen and Xiang Luo and Shen Yan and Qifan Yu and Fan Xia and Tianqi Zhang and Hongrui Zhan and Zheng Zhong and Xun Zhou and Siyuan Qiao and Xingyan Bin},
year = {2025},
eprint = {2510.24824},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2510.24824}
}