mlsanityData to 14 Oct 2026

RAMP: Recurrent Adaptive Mixed Precision for Efficient Prefill in Recurrent-Depth Language Models

arXiv 2610.13041 ↗First posted 9 Oct 2026cs.CLcited by 0

On the map: LLM reasoning → Diffusion reasoning and test-time scaling also fits: Test-time compute and efficient thinking

Abstract (arXiv)

Recurrent-depth language models improve inference-time reasoning by repeatedly applying a shared network to refine internal representations, but the additional recurrent computation also increases inference cost. This overhead is particularly pronounced during long-context prefill, where the full input sequence is repeatedly processed across recurrent depth. Weight-and-activation quantization offers a natural way to accelerate this computation through lower-precision arithmetic, yet repeatedly applying quantized computation introduces numerical perturbations that accumulate along the recurrent trajectory. We systematically analyze how such errors propagate across recurrent depth and show that quantization errors introduced earlier are attenuated by more subsequent recurrent steps and therefore have a smaller effect on the final representation. Matched-budget experiments strongly support this prediction, showing that placing quantized computation earlier in recurrent depth consistently yields the best fidelity. Based on this insight, we introduce RAMP (Recurrent Adaptive Mixed Precision), which performs early recurrent refinement using low-precision weight-and-activation computation and restores high precision for terminal refinement. Across context lengths up to 4K, RAMP preserves near-full-precision prefill quality while achieving up to 1.16x end-to-end first-prefill speedup on an NVIDIA H100, corresponding to a 13.9% latency reduction.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Diffusion reasoning and test-time scaling, one of 17 directions in LLM reasoning. Builds on 14 of its 16 references that are in the map. First posted on arXiv 9 Oct 2026.

Builds on

14 of 16 references.

References · 14

  1. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  2. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,298
  3. 2024QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMscited by 634
  4. 2024DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Servingcited by 617
  5. 2023Splitwise: Efficient generative LLM inference using phase splittingcited by 558
  6. 2024Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Servecited by 526
  7. 2024SpinQuant: LLM quantization with learned rotationscited by 471
  8. 2025Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approachcited by 415
  9. 2023Atom: Low-bit Quantization for Efficient and Accurate LLM Servingcited by 272
  10. 2025Reasoning with Latent Thoughts: On the Power of Looped Transformerscited by 240
  11. 2024QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Servingcited by 208
  12. 2026LoopFormer: Elastic-Depth Looped Transformers for Latent Reasoning via Shortcut Modulationcited by 61
2 more
  1. 2026Stabilizing Recurrent Dynamics for Test-Time Scalable Latent Reasoning in Looped Language Modelscited by 21
  2. 2026Attention Routing Stabilizes Early: Working-Set Inference for Recurrent Language Modelscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Efficient Parallel Samplers for Recurrent-Depth Models and Their Connection to Diffusion Language Models
  2. 2025Teaching Pretrained Language Models to Think Deeper with Retrofitted Recurrence
  3. 2026AdaPonderLM: Gated Pondering Language Models with Token-Wise Adaptive Depth
  4. 2026Recurrent-Depth VLA: Implicit Test-Time Compute Scaling of Vision-Language-Action Models via Latent Iterative Reasoning
  5. 2026DAMP: Decay-Aware Mixed-Precision Recurrent-State Quantization
  6. 2026RAMP: Reinforcement Adaptive Mixed Precision Quantization for Efficient On Device LLM Inference
  7. 2026Trading Depth for Time in Recurrent Transformers
  8. 2026RecurTrace: Adaptive Latent Reasoning with Loop-Time Memory