mlsanityData to 7 Oct 2026

The Benefits of a Concise Chain of Thought on Problem-Solving in Large Language Models

arXiv 2401.05618 ↗First posted 11 Jan 2024cs.CL cs.AIcited by 62

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

Abstract (arXiv)

In this paper, we introduce Concise Chain-of-Thought (CCoT) prompting. We compared standard CoT and CCoT prompts to see how conciseness impacts response length and correct-answer accuracy. We evaluated this using GPT-3.5 and GPT-4 with a multiple-choice question-and-answer (MCQA) benchmark. CCoT reduced average response length by 48.70% for both GPT-3.5 and GPT-4 while having a negligible impact on problem-solving performance. However, on math problems, GPT-3.5 with CCoT incurs a performance penalty of 27.69%. Overall, CCoT leads to an average per-token cost reduction of 22.67%. All code, data, and supplemental materials are available on GitHub at https://github.com/matthewrenze/jhu-concise-cot

Key facts. Cited by 62 papers in the map (data through 7 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 36 of its 57 references that are in the map. First posted on arXiv 11 Jan 2024.

Builds on

36 of 57 references.

Closest ancestors · 3

Same direction, newest first

  1. 2022Text and Patterns: For Effective Chain of Thought, It Takes Two to Tango16 Sep
  2. 2022Large Language Models are Zero-Shot Reasoners24 May
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Models28 Jan

Other references · 33

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  4. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  5. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  6. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  7. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  8. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  9. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  10. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
  11. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  12. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
21 more
  1. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
  2. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  3. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
  4. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,020
  5. 2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 965
  6. 2023AGIEval: A Human-Centric Benchmark for Evaluating Foundation Modelscited by 701
  7. 2023A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPTcited by 612
  8. 2023MemoryBank: Enhancing Large Language Models with Long-Term Memorycited by 539
  9. 2022MedMCQA : A Large-scale Multi-Subject Multi-Choice Dataset for Medical domain Question Answeringcited by 525
  10. 2023Augmented Language Models: a Surveycited by 398
  11. 2023Automatically Correcting Large Language Models: Surveying the landscape of diverse self-correction strategiescited by 236
  12. 2020Retrieve, Program, Repeat: Complex Knowledge Base Question Answering via Alternate Meta-learningcited by 216
  13. 2023SelfCheck: Using LLMs to Zero-Shot Check Their Own Step-by-Step Reasoningcited by 157
  14. 2023LLMs cannot find reasoning errors, but can correct them given the error locationcited by 85
  15. 2023Towards Mitigating Hallucination in Large Language Models via Self-Reflectioncited by 81
  16. 2023Metacognitive Prompting Improves Understanding in Large Language Modelscited by 58
  17. 2023Principled Instructions Are All You Need for Questioning LLaMA-1/2, GPT-3.5/4cited by 48
  18. 2021From LSAT: The Progress and Challenges of Complex Reasoningcited by 43
  19. 2023Agent Instructs Large Language Models to be General Zero-Shot Reasonerscited by 32
  20. 2023How susceptible are LLMs to Logical Fallacies?cited by 23
  21. 2024Metacognition is all you need? Using Introspection in Generative Agents to Improve Goal-directed Behaviorcited by 15

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 7; 2025: 42; 2026: 13.

62 papers.

Sort:

By area:

  1. 2025Stop Overthinking: A Survey on Efficient Reasoning for Large Language Modelscited by 456
  2. 2025rStar-Math: Small LLMs Can Master Math Reasoning with Self-Evolved Deep Thinkingcited by 308
  3. 2025Dynamic Early Exit in Reasoning Modelscited by 217
  4. 2024Judging the Judges: Evaluating Alignment and Vulnerabilities in LLMs-as-Judgescited by 197
  5. 2025Reasoning Models Can Be Effective Without Thinkingcited by 182
  6. 2024Faithfulness vs. Plausibility: On the (Un)Reliability of Explanations from Large Language Modelscited by 104
  7. 2025On Memory Construction and Retrieval for Personalized Conversational Agentscited by 101
  8. 2025How Well do LLMs Compress Their Own Chain-of-Thought? A Token Complexity Approachcited by 95
  9. 2025H-CoT: Hijacking the Chain-of-Thought Safety Reasoning Mechanism to Jailbreak Large Reasoning Models, Including OpenAI o1/o3, DeepSeek-R1, and Gemini 2.0 Flash Thinkingcited by 94
  10. 2025Efficient Reasoning Models: A Surveycited by 93
  11. 2025S-GRPO: Early Exit via Reinforcement Learning in Reasoning Modelscited by 87
  12. 2025AdaCoT: Pareto-Optimal Adaptive Chain-of-Thought Triggering via Reinforcement Learningcited by 86
38 more
  1. 2025Concise Reasoning via Reinforcement Learningcited by 78
  2. 2025RTBAS: Defending LLM Agents Against Prompt Injection and Privacy Leakagecited by 64
  3. 2025Learning When to Think: Shaping Adaptive Reasoning in R1-Style Models via Multi-Stage RLcited by 59
  4. 2025SelfBudgeter: Adaptive Token Allocation for Efficient LLM Reasoningcited by 47
  5. 2024A Theoretical Understanding of Self-Correction through In-context Alignmentcited by 43
  6. 2025Reflective Planning: Vision-Language Models for Multi-Stage Long-Horizon Robotic Manipulationcited by 40
  7. 2024Richelieu: Self-Evolving LLM-Based Agents for AI Diplomacycited by 36
  8. 2025Optimizing Model Selection for Compound AI Systemscited by 34
  9. 2025Reasoning on a Budget: A Survey of Adaptive and Controllable Test-Time Compute in LLMscited by 31
  10. 2025Let LRMs Break Free from Overthinking via Self-Braking Tuningcited by 27
  11. 2025Shorten After You're Right: Lazy Length Penalties for Reasoning RLcited by 25
  12. 2025Failure-Driven Workflow Refinementcited by 19
  13. 2025Towards Concise and Adaptive Thinking in Large Reasoning Models: A Surveycited by 18
  14. 2025OThink-R1: Intrinsic Fast/Slow Thinking Mode Switching for Over-Reasoning Mitigationcited by 17
  15. 2024MorphAgent: Empowering Agents through Self-Evolving Profiles and Decentralized Collaborationcited by 15
  16. 2025Speak Easy: Eliciting Harmful Jailbreaks from LLMs with Simple Interactionscited by 14
  17. 2025Can Aha Moments Be Fake? Towards Quantifying Decorative and True Thinking in Chain-of-Thoughtcited by 14
  18. 2024Think Together and Work Better: Combining Humans' and LLMs' Think-Aloud Outcomes for Effective Text Evaluationcited by 12
  19. 2025Evolutionary Perspectives on the Evaluation of LLM-Based AI Agents: A Comprehensive Surveycited by 12
  20. 2025Slim-SC: Thought Pruning for Efficient Scaling with Self-Consistencycited by 12
  21. 2025A*-Thought: Efficient Reasoning via Bidirectional Compression for Low-Resource Settingscited by 9
  22. 2025ConciseHint: Boosting Efficient Reasoning via Continuous Concise Hints during Generationcited by 9
  23. 2025Reasoning or Retrieval? A Study of Answer Attribution on Large Reasoning Modelscited by 9
  24. 2025Long or short CoT? Investigating Instance-level Switch of Large Reasoning Modelscited by 8
  25. 2025Aligning Instruction Tuning with Pre-trainingcited by 7
  26. 2025Scalable LLM Reasoning Acceleration with Low-rank Distillationcited by 7
  27. 2025AutoRAN: Automated Hijacking of Safety Reasoning in Large Reasoning Modelscited by 7
  28. 2025Evaluation of LLMs for mathematical problem solvingcited by 7
  29. 2025MemOrb: A Plug-and-Play Verbal-Reinforcement Memory Layer for E-Commerce Customer Servicecited by 7
  30. 2024From RAGs to riches: Utilizing large language models to write documents for clinical trialscited by 6
  31. 2025Why is prompting hard? Understanding prompts on binary sequence predictorscited by 4
  32. 2025Think How to Think: Mitigating Overthinking with Autonomous Difficulty Cognition in Large Reasoning Modelscited by 4
  33. 2025MixReasoning: Switching Modes to Thinkcited by 3
  34. 2025Q-NL Verifier: Leveraging Synthetic Data for Robust Knowledge Graph Question Answeringcited by 2
  35. 2025Creativity or Brute Force? Using Brainteasers as a Window into the Problem-Solving Abilities of Large Language Modelscited by 2
  36. 2025Cross-Question Method Reuse in Large Language Models: From Word-Level Prediction to Rational Logical-Layer Reasoningcited by 2
  37. 2026SAT: Balancing Reasoning Accuracy and Efficiency with Stepwise Adaptive Thinkingcited by 2
  38. 2026Does Accuracy Equal Evidence? Reasoning Faithfulness under KV Cache Compressioncited by 2

50 most cited of 62 All 62 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023Stress Testing Chain-of-Thought Prompting for Large Language Models
  2. 2023Towards Better Chain-of-Thought Prompting Strategies: A Survey
  3. 2025Prompting Science Report 2: The Decreasing Value of Chain of Thought in Prompting
  4. 2022Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matters
  5. 2024Enhancing Chain of Thought Prompting in Large Language Models via Reasoning Patterns
  6. 2024Boosting of Thoughts: Trial-and-Error Problem Solving with Large Language Models
  7. 2024Towards Better Chain-of-Thought: A Reflection on Effectiveness and Faithfulness
  8. 2024Concise Thoughts: Impact of Output Length on LLM Reasoning and Cost