mlsanityData to 14 Oct 2026

Shallow Queries, Mature Values: Depth-Asynchronous Self-Speculation for Looped Transformers

arXiv 2609.34538 ↗First posted 28 Sep 2026cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Looped Transformers reuse a shared block across recurrent depths, making autoregressive decoding expensive because every generated token requires many sequential recurrent passes. Self-speculative decoders reduce this cost by drafting at an early depth and verifying at full depth, but typically bind draft computation to prefix representations from the same recurrent depth. We find that queries and keys approach their final-depth representations earlier than values, and controlled prefix-channel interventions show that mature values substantially improve shallow draft predictions. Motivated by this asymmetry, we introduce Depth-Asynchronous Self-Speculation (DAS), which decouples the depth of draft computation from the depth of verified-prefix representations it reads. Its Mature-V primitive lets shallow queries retrieve full-depth prefix values without additional recurrent computation. We further develop DAS-Wave, which combines depth-asynchronous prefix reads with carried parallel refinement, progressive block growth, and an independent full-depth verifier. Across four recurrent-model checkpoints and mathematics and code workloads, DAS-Wave achieves 4.00--6.96× mean throughput speedup over paired full-depth autoregressive decoding in the same inference stack. These results identify prefix-information depth as an effective design axis for recurrent self-speculation.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 21 of its 23 references that are in the map. First posted on arXiv 28 Sep 2026.

Builds on

21 of 23 references.

References · 21

  1. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  3. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  4. 2023Let's Verify Step by Stepcited by 3,789
  5. 2021Program Synthesis with Large Language Modelscited by 3,637
  6. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,653
  7. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,392
  8. 2023Accelerating Large Language Model Decoding with Speculative Samplingcited by 1,006
  9. 2025Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approachcited by 415
  10. 2024LayerSkip: Enabling Early Exit Inference and Self-Speculative Decodingcited by 307
  11. 2024Break the Sequential Dependency of LLM Inference Using Lookahead Decodingcited by 298
  12. 2023Looped Transformers as Programmable Computerscited by 237
9 more
  1. 2024Relaxed Recursive Transformers: Effective Parameter Sharing with Layer-wise LoRAcited by 78
  2. 2023SPEED: Speculative Pipelined Execution for Efficient Decodingcited by 47
  3. 2024Turning Trash into Treasure: Accelerating Inference of Large Language Models with Token Recyclingcited by 47
  4. 2025Teaching Pretrained Language Models to Think Deeper with Retrofitted Recurrencecited by 47
  5. 2024SAM Decoding: Speculative Decoding via Suffix Automatoncited by 39
  6. 2025SCOUT: Teaching Pre-trained Language Models to Enhance Reasoning via Flow Chain-of-Thoughtcited by 7
  7. 2025Efficient Parallel Samplers for Recurrent-Depth Models and Their Connection to Diffusion Language Modelscited by 7
  8. 2026Depth-adaptive Inference of Looped Language Models via Continuous Depth Batchingcited by 6
  9. 2026LoopSpec: Pipelined Self-Speculative Decoding for Looped Transformerscited by 2

Built on by

Not cited yet.