IAAIQuantizing Looped Transformers
Quantizing Looped Transformers: Feedback Exposure and Calibration Blindness
Identifies two failure modes of post-training quantization in looped models: feedback exposure, where a quantized non-residual layer such as the loop-entry adapter perturbs the state and the error is fed back, and calibration blindness, where one-step calibration ignores directions used later in the recurrence.
Inside the method
Simplified conceptual schematic. Consult the paper for the complete architecture.
- Recurrence family
- Core recurrence
- Depth control
- Fixed recurrence
- KV / state strategy
- See paper
Reading note
On Huginn-3.5B, INT4 fails mainly at the loop-entry adapter. Across nine checkpoints from seven looped architectures, accumulating the GPTQ Hessian across recurrence steps beats one-step GPTQ and RTN everywhere and recovers bf16-level accuracy on Huginn.
Sources checked 2026-10-28. This catalog does not imply independent reproduction.
Related works
Cite this work
@misc{quantizingloopedtransformers2026,
title = {Quantizing Looped Transformers: Feedback Exposure and Calibration Blindness},
author = {Nux Li},
year = {2026},
eprint = {2609.30820},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.30820}
}