mlsanityData to 7 Oct 2026

ConCISE: Confidence-guided Compression in Step-by-step Efficient Reasoning

arXiv 2505.04881 ↗First posted 8 May 2025cs.LG cs.AI cs.CLcited by 33

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Large Reasoning Models (LRMs) perform strongly in complex reasoning tasks via Chain-of-Thought (CoT) prompting, but often suffer from verbose outputs, increasing computational overhead. Existing fine-tuning-based compression methods either operate post-hoc pruning, risking disruption to reasoning coherence, or rely on sampling-based selection, which fails to remove redundant content thoroughly. To address these limitations, this work begins by framing two key patterns of redundant reflection in LRMs--Confidence Deficit, wherein the model reflects on correct intermediate steps, and Termination Delay, where reflection continues after a verified, confident answer--through a confidence-guided perspective. Based on this, we introduce ConCISE (Confidence-guided Compression In Step-by-step Efficient Reasoning), a framework designed to generate concise reasoning chains, integrating Confidence Injection to boost reasoning confidence, and Early Stopping to terminate reasoning when confidence is sufficient. Extensive experiments demonstrate that compared to baseline methods, fine-tuning LRMs on ConCISE-generated data yields a better balance between compression and task performance, reducing length by up to approximately 50% under SimPO, while maintaining high task accuracy.

Key facts. Cited by 33 papers in the map (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 35 of its 45 references that are in the map. First posted on arXiv 8 May 2025.

Builds on

35 of 45 references.

Closest ancestors · 17

Same direction, newest first

  1. 2025Dynamic Early Exit in Reasoning Models22 Apr
  2. 2025Efficient Reasoning Models: A Survey15 Apr
  3. 2025A Survey of Efficient Reasoning for Large Reasoning Models: Language, Multimodality, and Beyond27 Mar
  4. 2025Stop Overthinking: A Survey on Efficient Reasoning for Large Language Models20 Mar
  5. 2025L1: Controlling How Long A Reasoning Model Thinks With Reinforcement Learning6 Mar
  6. 2025DAST: Difficulty-Adaptive Slow-Thinking for Large Reasoning Models6 Mar
  7. 2025How Well do LLMs Compress Their Own Chain-of-Thought? A Token Complexity Approach3 Mar
  8. 2025Self-Training Elicits Concise Reasoning in Large Language Models27 Feb
  9. 2025LightThinker: Thinking Step-by-Step Compression21 Feb
  10. 2025TokenSkip: Controllable Chain-of-Thought Compression in LLMs17 Feb
  11. 2025CoT-Valve: Length-Compressible Chain-of-Thought Tuning13 Feb
  12. 2025s1: Simple test-time scaling31 Jan
5 more
  1. 2025Thoughts Are All Over the Place: On the Underthinking of o1-Like LLMs30 Jan
  2. 2025O1-Pruner: Length-Harmonizing Fine-Tuning for O1-Like Reasoning Pruning22 Jan
  3. 2024Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMs30 Dec
  4. 2024Token-Budget-Aware LLM Reasoning24 Dec
  5. 2024Concise Thoughts: Impact of Output Length on LLM Reasoning and Cost29 Jul

Other references · 18

  1. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  3. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  4. 2024Qwen2.5 Technical Reportcited by 4,713
  5. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  6. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,678
  7. 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,442
  8. 2024OpenAI o1 System Cardcited by 1,220
  9. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,193
  10. 2024Training Large Language Models to Reason in a Continuous Latent Spacecited by 763
  11. 2025Chain of Draft: Thinking Faster by Writing Lesscited by 290
  12. 2025Training Language Models to Reason Efficientlycited by 242
6 more
  1. 2025CODI: Compressing Chain-of-Thought into Continuous Space via Self-Distillationcited by 216
  2. 2025When More is Less: Understanding Chain-of-Thought Length in LLMscited by 182
  3. 2025Sketch-of-Thought: Efficient LLM Reasoning with Adaptive Cognitive-Inspired Sketchingcited by 116
  4. 2024Fast Best-of-N Decoding via Speculative Rejectioncited by 112
  5. 2025Stepwise Perplexity-Guided Refinement for Efficient Chain-of-Thought Reasoning in Large Language Modelscited by 51
  6. 2025Confident or Seek Stronger: Exploring Uncertainty-Based On-device LLM Routing From Benchmarking to Generalizationcited by 29

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 14; 2026: 19.

33 papers.

Sort:

By area:

  1. 2025S-GRPO: Early Exit via Reinforcement Learning in Reasoning Modelscited by 87
  2. 2025Don't Overthink It: A Survey of Efficient R1-style Large Reasoning Modelscited by 36
  3. 2025R1-Compress: Long Chain-of-Thought Compression via Chunk Compression and Searchcited by 29
  4. 2025Towards Concise and Adaptive Thinking in Large Reasoning Models: A Surveycited by 18
  5. 2026Does Your Reasoning Model Implicitly Know When to Stop Thinking?cited by 17
  6. 2025Stop When Enough: Adaptive Early-Stopping for Chain-of-Thought Reasoningcited by 15
  7. 2025Aware First, Think Less: Dynamic Boundary Self-Awareness Drives Extreme Reasoning Efficiency in Large Language Modelscited by 13
  8. 2026Self-Compression of Chain-of-Thought via Multi-Agent Reinforcement Learningcited by 13
  9. 2025LAPO: Internalizing Reasoning Efficiency via Length-Adaptive Policy Optimizationcited by 11
  10. 2025Hierarchical Budget Policy Optimization for Adaptive Reasoningcited by 8
  11. 2026VTC-R1: Vision-Text Compression for Efficient Long-Context Reasoningcited by 8
  12. 2025BadThink: Triggered Overthinking Attacks on Chain-of-Thought Reasoning in Large Language Modelscited by 7
21 more
  1. 2026Neural Chain-of-Thought Search: Searching the Optimal Reasoning Path to Enhance Large Language Modelscited by 7
  2. 2026Mitigating Overthinking in Large Reasoning Language Models via Reasoning Path Deviation Monitoringcited by 6
  3. 2025PEAR: Phase Entropy Aware Reward for Efficient Reasoningcited by 5
  4. 2026When Is Thinking Enough? Early Exit via Sufficiency Assessment for Efficient Reasoningcited by 5
  5. 2025From "Aha Moments" to Controllable Thinking: Toward Meta-Cognitive Reasoning in Large Reasoning Models via Decoupled Reasoning and Controlcited by 4
  6. 2026Compress the Easy, Explore the Hard: Difficulty-Aware Entropy Regularization for Efficient LLM Reasoningcited by 4
  7. 2025Verifying Meta-Awareness via Predictive Rewards in Reasoning Modelscited by 3
  8. 2025Learning to Reason Efficiently with Discounted Reinforcement Learningcited by 3
  9. 2025When Reasoning Meets Its Lawscited by 3
  10. 2026Steering Large Reasoning Models towards Concise Reasoning via Flow Matchingcited by 3
  11. 2026CoRefine: Confidence-Guided Self-Refinement for Adaptive Test-Time Computecited by 3
  12. 2026Stop Unnecessary Reflection: Training LRMs for Efficient Reasoning with Adaptive Reflection and Length Coordinated Penaltycited by 2
  13. 2026Learning to Interrupt in Language-based Multi-agent Communicationcited by 2
  14. 2026BFS-PO: Best-First Search for Large Reasoning Modelscited by 1
  15. 2026ThoughtFold: Folding Reasoning Chains via Introspective Preference Learningcited by 1
  16. 2026BLUE: Toward Better Language Use in Efficient Vision-Language-Action Models for Autonomous Drivingcited by 1
  17. 2026Demystifying Entropy-based Selection for Chain-of-Thought Compression in Large Reasoning Modelscited by 1
  18. 2026Mitigating Cognitive Inertia in Large Reasoning Models via Latent Spike Steeringcited by 0
  19. 2026Precedent-Informed Reasoning: Mitigating Overthinking in Large Reasoning Models via Test-Time Precedent Learningcited by 0
  20. 2026PathCal: State-Aware Reflection-Marker Calibration for Efficient Reasoningcited by 0
  21. 2026Learning to Stop without Learning to Stop: Self-Supervised Confidence Training Improves Reasoning Efficiencycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Optimizing Length Compression in Large Reasoning Models
  2. 2026ConMax: Confidence-Maximizing Compression for Efficient Chain-of-Thought Reasoning
  3. 2026Think Less, Know More: State-Aware Reasoning Compression with Knowledge Guidance for Efficient Reasoning
  4. 2025When Reasoning Meets Compression: Understanding the Effects of LLMs Compression on Large Reasoning Models
  5. 2026CAT: Confidence-Adaptive Thinking for Efficient Reasoning of Large Reasoning Models
  6. 2026Compress-Distill: Reasoning Trace Compression for Efficient Knowledge Distillation
  7. 2026Reasoning Compression with Mixed-Policy Distillation
  8. 2025Reasoning Path Compression: Compressing Generation Trajectories for Efficient LLM Reasoning