IAAIReadout Blind Spot
Dense Supervision Is Not Enough: The Readout Blind Spot in Looped Language Models
Shows that dense per-loop cross-entropy only controls state variables visible to the readout. Scale-invariant readouts such as RMSNorm hide radial scale from the loss while pre-norm residual recurrence keeps growing it, so early exits work but recurrent scale goes unchecked.
Inside the method
Simplified conceptual schematic. Consult the paper for the complete architecture.
- Recurrence family
- Full-stack recurrence
- Depth control
- Variable inference depth
- KV / state strategy
- See paper
Reading note
In 44M and 129M looped models without inter-loop normalization, hidden-state norms reach thousands or tens of thousands; scale-visible readouts, norm penalties or scale-removing recurrence keep them in the tens and lower perplexity at matched depth.
Sources checked 2026-10-28. This catalog does not imply independent reproduction.
Related works
Cite this work
@misc{readoutblindspot2026,
title = {Dense Supervision Is Not Enough: The Readout Blind Spot in Looped Language Models},
author = {Rituraj Sharma and Tu Vu},
year = {2026},
eprint = {2606.24898},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2606.24898}
}