Think deeper.
Find your Aha.

Recurrent Denoiser

Diffusion model with recurrent memory that does more than state-based self-correction

SMELT

Repeats the middle half of a sparse MoE Transformer's layers twice while closely matching per-token FLOPs, total non-embedding parameters and KV cache to an unlooped baseline. Budget-matched scaling experiments show improved loss and downstream performance from reusing the shared middle layers.

Nanbeige4.2-3B

A released language model that applies recurrent computation across a shared decoder stack.

DeepLoop

Revisits residual scaling for repeated parameter use, focusing on stable training of deeper looped networks.

ERM

Energy-guided Recursive Model (ERM) augments recurrent reasoning with an explicit energy-based mechanism for selecting among candidate trajectories. It constructs task-structured Hopfield energies over valid local or global patterns, using them to rank recurrent outputs and to support energy-guided sampling such as parallel tempering.

LOTUS

Trains parallel latent blocks with explicit reasoning-step supervision to compress the thought phase.

FPRM

Studies stable deep recurrence and fixed-point convergence as a stopping signal for structured reasoning.

Residual Scaling

Analyzes how weight sharing correlates residual updates across loop iterations and derives a 1/N residual scale for a block repeated N times. Factoring loop count from unique-layer depth improves training stability and supports learning-rate transfer across loop counts.

dvlt

The Loop transformer in 3D

LT²

Combines looping with linear, sparse and hybrid token mixers to explore more efficient repeated computation.

EqR

Equilibrium reasoners proposes a new view of recurrence as learning stable attractors as opposed to fixed point convergence. The new framework allows for more sophisticated test time scaling, and ultimately leads to stronger performance compared to the base TRM model.

LDT

The lattice deduction transformer is a looped transformer that projects the token space through a lattice structure to perform efficient combinatorial reasoning.

Parcae

Uses stable state injection to examine the scaling of looped Transformers with recurrence and training data.

SpiralFormer

Explores recurrent computation at different sequence resolutions to change the cost of repeated processing.

LoopFormer

Conditions shared computation on time and step size to support elastic inference budgets.

Think-at-Hard

Allocates extra latent iterations to selected tokens, using a decider and duo-causal attention.

Ouro

Revisits hidden states with a shared decoder stack over multiple latent computation passes.

TRM

The tiny recursive model is a looped transformer architecture with an intentionally simple architecture, yet yields strong generalization compared to previous models.

Mixture-of-Recursions

Routes tokens through different numbers of shared recursions and studies recursion-aware caching strategies.

Huginn

A recurrent core refines hidden states between a prelude and a coda, scaling latent computation at inference time.

Relaxed Recursive Transformers

Converts pretrained Transformers into recursive models, relaxing sharing with depth-specific low-rank adaptations.

MoEUT

Combines the Universal Transformer with mixture-of-experts layers to study shared depth and sparse capacity.

Learning to Learn with Loops

Studies how looping encourages Transformers to learn iterative learning algorithms in context.

Sliced-Recursive-Transformer (SReT)

This paper presents a neat yet effective recursive operation on transformers that can improve parameter utilization without involving additional parameters. This is achieved by sharing weights across the depth of transformer networks. The proposed method can obtain a substantial gain (~2%) simply using a naive recursive operation, requires no special or sophisticated knowledge for designing principles of networks, and introduces minimal computational overhead to the training procedure. To reduce the additional computation caused by recursive operation while maintaining the superior accuracy, we propose an approximation method through multiple sliced group self-attentions across recursive layers, which can reduce the cost consumption by 10~30% with minimal performance loss.

Universal Transformers

Shares a transformation across depth, with adaptive computation time allowing each position to halt independently.

Recurrent Denoiser

Recurrence

See paper for architecture

Depth control

See paper

KV / state strategy

See paper

Reading note

Community-submitted work. Consult the linked paper for architecture details and experimental conditions.

SMELT

Recurrence

Front layers → [ Middle half ] × 2 → Back layers

Depth control

Fixed: middle half of layers repeated twice

KV / state strategy

Matched to the unlooped baseline

Reading note

Separate fitted Chinchilla-style scaling laws estimate 6.8–18.0% training-FLOP savings on the compute-optimal frontier. Experiments cover four sizes up to 54B non-embedding parameters, with the largest domain gains on Code and stronger gains for longer samples and more in-context examples.

Nanbeige4.2-3B

Recurrence

[ 22-layer decoder stack ] × 2

Depth control

Two passes in released config

KV / state strategy

See model implementation

Reading note

The published configuration specifies 22 layers and 2 loops. Other modes supported by the code do not imply that the released weights use them.

DeepLoop

Recurrence

[ Residual-scaled shared layers ] × R

Depth control

Deep recurrence

KV / state strategy

See paper

Reading note

No public weights were confirmed in this review.

ERM

Recurrence

EqR-style recurrent reasoner × D → K candidate trajectories → Hopfield-energy selector → selected answer

Depth control

D=64 recurrent steps in the main experiments; test-time computation scales through recurrent depth and candidate breadth

KV / state strategy

See paper

Reading note

ERM primarily changes the inference-time selection mechanism rather than replacing the recurrent reasoning backbone. The main experiments use EqR-style recurrent rollouts with D=64 and K=128 candidates. The energy is task-structured: Sudoku uses row, column, and box permutation memories, PPBench uses puzzle-specific local memories and global distance potentials, and the reported Maze global-memory setting uses an input-conditioned shortest-path memory.

LOTUS

Recurrence

[ Parallel latent blocks ] × R → Answer

Depth control

Latent iteration budget

KV / state strategy

Fixed prefix context

Reading note

Reported thought-phase latency improvements are not equivalent to end-to-end speedups for all requests.

FPRM

Recurrence

[ State refinement ] → Fixed point

Depth control

Convergence-based halting

KV / state strategy

Task / variant dependent

Reading note

The public checkpoint collection includes convolutional variants. Check the specific architecture before equating it with the paper’s Transformer.

Residual Scaling

Recurrence

[ Residual-scaled shared block ] × N

Depth control

Configurable N loops over L unique layers

KV / state strategy

See paper

Reading note

For L unique layers repeated N times, the parameterization is epsilon = lambda / (N sqrt(L)). The analysis separates correlation across loops from variance across unique layers; Transformer experiments compare loss and learning-rate transfer across loop counts at fixed unique-layer depth.

dvlt

Recurrence

See paper for architecture

Depth control

See paper

KV / state strategy

See paper

Reading note

Community-submitted work. Consult the linked paper for architecture details and experimental conditions.

LT²

Recurrence

[ Linear / sparse / hybrid mixer ] × R

Depth control

Iteration budget

KV / state strategy

Mixer-dependent state

Reading note

The Full+GDN variant still includes full attention; not all variants have strictly linear attention complexity.

EqR

Recurrence

See paper for architecture

Depth control

See paper

KV / state strategy

See paper

Reading note

Community-submitted work. Consult the linked paper for architecture details and experimental conditions.

LDT

Recurrence

See paper for architecture

Depth control

See paper

KV / state strategy

See paper

Reading note

Community-submitted work. Consult the linked paper for architecture details and experimental conditions.

Parcae

Recurrence

Prelude → [ Stable core ] × R → Coda

Depth control

Variable recurrent depth

KV / state strategy

See implementation

Reading note

The repository provides entry points for 140M, 370M, 770M and 1.3B pretrained models. Availability is not a local reproduction result.

SpiralFormer

Recurrence

[ Multi-resolution states ] × R

Depth control

Multi-resolution recurrence

KV / state strategy

See paper

Reading note

No official code or weights were confirmed in this review. This entry is not the similarly named speech model.

LoopFormer

Recurrence

h(t) → F(t, Δt) → h(t+Δt)

Depth control

Budget-conditioned depth

KV / state strategy

See implementation

Reading note

Adjustable budgets are distinct from automatically deciding when every token should stop.

Think-at-Hard

Recurrence

Token → Decider → Latent iterations

Depth control

Selective token iteration

KV / state strategy

Duo-causal attention

Reading note

Selective additional iteration and ordinary layer skipping have different computational semantics.

Ouro

Recurrence

[ Decoder stack ] × R

Depth control

Loop count + exit policy

KV / state strategy

Per-loop cache

Reading note

The inspected Ouro-2.6B implementation computes all loop states before selecting an exit output. Its exit option alone does not demonstrate skipped computation. Published configuration: 48 layers, 4 passes.

TRM

Recurrence

See paper for architecture

Depth control

See paper

KV / state strategy

See paper

Reading note

Community-submitted work. Consult the linked paper for architecture details and experimental conditions.

Mixture-of-Recursions

Recurrence

Token → Router → Rᵢ loops

Depth control

Token-level routing

KV / state strategy

Recursion-wise / shared KV

Reading note

Appendix E measures forward time without KV cache maintenance. Do not read its throughput gains as universal end-to-end serving speedups.

Huginn

Recurrence

Prelude → [ Core ] × R → Coda

Depth control

Variable recurrent depth

KV / state strategy

Recurrence-aware cache

Reading note

Huginn-0125 uses 2 prelude layers, 4 shared core layers and 2 coda layers. At mean recurrence 32, expected effective depth is 132. Independent parameters: about 3.5B.

Relaxed Recursive Transformers

Recurrence

[ Shared layers + depth LoRA ] × R

Depth control

Recursion budget

KV / state strategy

Continuous depth-wise batching

Reading note

The reported batching throughput gains are based on analysis and simulation; distinguish them from end-to-end serving measurements.

MoEUT

Recurrence

[ Shared attention + MoE ] × R

Depth control

See paper

KV / state strategy

See paper

Reading note

Code and checkpoint availability have not been audited for this entry.

Learning to Learn with Loops

Recurrence

[ Shared Transformer ] × R

Depth control

Iteration budget

KV / state strategy

Task-specific experimental setup

Reading note

A controlled algorithm-learning study, rather than a general-purpose chat model.

Sliced-Recursive-Transformer (SReT)

Recurrence

See paper for architecture

Depth control

See paper

KV / state strategy

See paper

Reading note

Community-submitted work. Consult the linked paper for architecture details and experimental conditions.

Universal Transformers

Recurrence

Shared transformation + ACT

Depth control

Per-position adaptive halting

KV / state strategy

Depth-wise shared self-attention

Reading note

The foundation for recurrent depth and position-wise adaptive computation.