IAAICausal Latent Workspace
Causal Latent Workspaces for Looped Transformers
Gives a shared-depth Transformer a dedicated causal workspace: at each of four rounds, positions write gated vectors into prefix-specific latent slots and read back a content-addressed summary before the next application of the shared block, while the input embedding stays accessible.
Inside the method
[ Shared block + causal latent workspace ] × 4 (input anchored)
Simplified conceptual schematic. Consult the paper for the complete architecture.
- Recurrence family
- Full-stack recurrence
- Depth control
- Four fixed rounds
- KV / state strategy
- Gated prefix-specific latent slots with content-addressed reads, alongside causal self-attention
Reading note
On three byte-level corpora at about 1M and 101.71M parameters, it has the best mean held-out perplexity in five of six settings against five controls, with roughly 1% gains at the larger budget. Small-scale study.
Sources checked 2026-10-28. This catalog does not imply independent reproduction.
Related works
Cite this work
@misc{causallatentworkspace2026,
title = {Causal Latent Workspaces for Looped Transformers},
author = {Ilia Zaznov},
year = {2026},
eprint = {2610.22796},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.22796}
}