mlsanityData to 7 Oct 2026

Prefix Sliding for efficient test-time scaling

arXiv 2608.26070 ↗First posted 26 Aug 2026cs.CL cs.AI cs.LGcited by 2

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Test-time scaling uses extra test-time compute to improve performance, such as letting language models reason longer when solving a problem. As models keep the entire reasoning trace in memory via full attention, hard tasks that need long thinking can be prohibitively expensive. However, we find most intermediate reasoning tokens lose importance as the model continues reasoning. This calls into question whether retaining them is worth the cost. Based on this insight, we propose Prefix Sliding, which discards tokens during reasoning that are not part of the prefix or the window of the last few thousand tokens. The prefix has key instructions and tools available to the model, while the most recent tokens are the current reasoning the model is working on. This caps the total memory requirement regardless of how long the model reasons, allowing for efficient long-horizon test-time scaling. Without training, Prefix Sliding can make existing models 3x faster while maintaining performance. Training with Prefix Sliding using reinforcement learning can achieve better performance by enabling scaling to reasoning traces beyond a hundred thousand tokens. Ablations show Prefix Sliding outperforms summarizing intermediate tokens or vanilla sliding window. Our code is at https://github.com/Muennighoff/prefix-sliding

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 137 of its 154 references that are in the map. First posted on arXiv 26 Aug 2026.

Builds on

137 of 154 references.

Closest ancestors · 32

Same direction, newest first

  1. 2026Value-Aware Stochastic KV Cache Eviction for Reasoning Models2 Jun
  2. 2026KARA: Efficient Reasoning LLM Serving via Sliding-Window KV Cache Compression1 May
  3. 2026MEMENTO: Teaching LLMs to Manage Their Own Context10 Apr
  4. 2026InftyThink+: Effective and Efficient Infinite-Horizon Reasoning via Reinforcement Learning6 Feb
  5. 2026Thinking Long, but Short: Stable Sequential Test-Time Scaling for Large Reasoning Models14 Jan
  6. 2026Crystal-KV: Efficient KV Cache Management for Chain-of-Thought LLMs via Answer-First Principle5 Jan
  7. 2025Recursive Language Models31 Dec
  8. 2025Wait, Wait, Wait... Why Do Reasoning Models Loop?15 Dec
  9. 2025ParoQuant: Pairwise Rotation Quantization for Efficient Reasoning LLM Inference13 Nov
  10. 2025The Markovian Thinker: Architecture-Agnostic Linear Scaling of Reasoning8 Oct
  11. 2025ThinKV: Thought-Adaptive KV Cache Compression for Efficient Reasoning Models1 Oct
  12. 2025gpt-oss-120b & gpt-oss-20b Model Card8 Aug
20 more
  1. 2025Inverse Scaling in Test-Time Compute19 Jul
  2. 2025Steering LLM Thinking with Budget Guidance16 Jun
  3. 2025Kinetics: Rethinking Test-Time Scaling Laws5 Jun
  4. 2025Just Enough Thinking: Efficient Reasoning with Adaptive Length Penalties Reinforcement Learning5 Jun
  5. 2025R-KV: Redundancy-aware KV Cache Compression for Reasoning Models30 May
  6. 2025Thought calibration: Efficient and confident test-time scaling23 May
  7. 2025Let LRMs Break Free from Overthinking via Self-Braking Tuning20 May
  8. 2025Scalable LLM Reasoning Acceleration with Low-rank Distillation8 May
  9. 2025Retro-Search: Exploring Untaken Paths for Deeper and Efficient Reasoning6 Apr
  10. 2025ThinkPrune: Pruning Long Chain-of-Thought of LLMs via Reinforcement Learning2 Apr
  11. 2025A Survey on Test-Time Scaling in Large Language Models: What, How, Where, and How Well?31 Mar
  12. 2025InftyThink: Breaking the Length Limits of Long-Context Reasoning in Large Language Models9 Mar
  13. 2025L1: Controlling How Long A Reasoning Model Thinks With Reinforcement Learning6 Mar
  14. 2025DAST: Difficulty-Adaptive Slow-Thinking for Large Reasoning Models6 Mar
  15. 2025RaaS: Reasoning-Aware Attention Sparsity for Efficient LLM Reasoning16 Feb
  16. 2025s1: Simple test-time scaling31 Jan
  17. 2025O1-Pruner: Length-Harmonizing Fine-Tuning for O1-Like Reasoning Pruning22 Jan
  18. 2024LightTransfer: Your Long-Context LLM is Secretly a Hybrid Model with Effortless Adaptation17 Oct
  19. 2024Learning How Hard to Think: Input-Adaptive Allocation of LM Computation7 Oct
  20. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters6 Aug

Other references · 105

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  3. 2025Qwen3 Technical Reportcited by 8,195
  4. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  5. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  6. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  7. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
  8. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  9. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
  10. 2023Let's Verify Step by Stepcited by 3,598
  11. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  12. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
38 more
  1. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,678
  2. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,424
  3. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,115
  4. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,838
  5. 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,830
  6. 2020Big Bird: Transformers for Longer Sequencescited by 1,673
  7. 2020Reformer: The Efficient Transformercited by 1,658
  8. 2020Rethinking Attention with Performerscited by 1,584
  9. 2019Generating Long Sequences with Sparse Transformerscited by 1,523
  10. 2020Linformer: Self-Attention with Linear Complexitycited by 1,462
  11. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,338
  12. 2024Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Dualitycited by 1,283
  13. 2025Understanding R1-Zero-Like Training: A Critical Perspectivecited by 1,276
  14. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,234
  15. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,193
  16. 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,099
  17. 2024OpenHands: An Open Platform for AI Software Developers as Generalist Agentscited by 956
  18. 2025Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?cited by 948
  19. 2024Large Language Monkeys: Scaling Inference Compute with Repeated Samplingcited by 946
  20. 2025OpenAI GPT-5 System Cardcited by 910
  21. 2024SnapKV: LLM Knows What You are Looking for Before Generationcited by 848
  22. 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Modelscited by 805
  23. 2025DeepSeek-V3.2: Pushing the Frontier of Open Large Language Modelscited by 755
  24. 2023AGIEval: A Human-Centric Benchmark for Evaluating Foundation Modelscited by 701
  25. 2019Compressive Transformers for Long-Range Sequence Modellingcited by 699
  26. 2020Long Range Arena: A Benchmark for Efficient Transformerscited by 696
  27. 2023Measuring Faithfulness in Chain-of-Thought Reasoningcited by 668
  28. 2017Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problemscited by 642
  29. 2023Gated Linear Attention Transformers with Hardware-Efficient Trainingcited by 490
  30. 2024Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inferencecited by 487
  31. 2021Nystr"omformer: A Nystr"om-Based Algorithm for Approximating Self-Attentioncited by 468
  32. 2023Model Tells You What to Discard: Adaptive KV Cache Compression for LLMscited by 466
  33. 2023Ring Attention with Blockwise Transformers for Near-Infinite Contextcited by 448
  34. 2023Scissorhands: Exploiting the Persistence of Importance Hypothesis for LLM KV Cache Compression at Test Timecited by 428
  35. 2024PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funnelingcited by 420
  36. 2025Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attentioncited by 391
  37. 2025Reasoning Models Don't Always Say What They Thinkcited by 388
  38. 2025HealthBench: Evaluating Large Language Models Towards Improved Human Healthcited by 385

50 most cited of 105 All 105 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Random Attention: Rethinking KV Cache Eviction for Efficient Reasoningcited by 1
  2. 2026KV-streams for Efficient Compaction in Agentic Reinforcement Learningcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2025Extending Test-Time Scaling: A 3D Perspective with Context, Batch, and Turn
  2. 2026Efficient Test-Time Scaling via Temporal Reasoning Aggregation
  3. 2025Learning a Continue-Thinking Token for Enhanced Test-Time Scaling
  4. 2025Step-level Verifier-guided Hybrid Test-Time Scaling for Large Language Models
  5. 2025On the Role of Temperature Sampling in Test-Time Scaling
  6. 2026Test-Time Scaling in Reasoning LLMs: Inference Regimes, Evaluation, and Reproducibility
  7. 2025Maximizing Prefix-Confidence at Test-Time Efficiently Improves Mathematical Reasoning
  8. 2025Rethinking Fine-Tuning when Scaling Test-Time Compute: Limiting Confidence Improves Mathematical Reasoning