mlsanityData to 7 Oct 2026

Concise Thoughts: Impact of Output Length on LLM Reasoning and Cost

arXiv 2407.19825 ↗First posted 29 Jul 2024cs.CL cs.AIcited by 108

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Today's large language models (LLMs) can solve challenging question-answering tasks, and prompt engineering techniques, such as chain-of-thought (CoT), have gained attention for enhancing the explanation and correctness of outputs. However, many models and techniques tend to produce excessively verbose and lengthy answers, leading to issues with both conciseness and generation time. To address this, this paper analyzes the impact of output lengths on LLM inference pipelines by introducing and proposing novel metrics to evaluate the textitcorrect conciseness of a model and related prompting techniques. Then, we examine the impact of controlling output length through a refined prompt engineering strategy, Constrained-CoT (CCoT), which encourages the model to produce more concise outputs. To better understand the effects of such a prompt, we also introduce two additional scores for analyzing the conciseness, measured in terms of redundancy and information flow in generated answers. Experiments on pretrained LLMs and multiple datasets demonstrate the benefits of the proposed metrics and the effectiveness of CCoT across different models.

Key facts. Cited by 108 papers in the map (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 28 of its 43 references that are in the map. First posted on arXiv 29 Jul 2024.

Builds on

28 of 43 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022Language Models (Mostly) Know What They Know11 Jul

Other references · 27

  1. 2017Attention Is All You Needcited by 59,932
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  4. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  5. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  6. 2020Scaling Laws for Neural Language Modelscited by 6,411
  7. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  8. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  9. 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
  10. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,538
  11. 2017Get To The Point: Summarization with Pointer-Generator Networkscited by 1,281
  12. 2019How Can We Know What Language Models Know?cited by 953
15 more
  1. 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 820
  2. 2023The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Onlycited by 755
  3. 2023The Internal State of an LLM Knows When It's Lyingcited by 644
  4. 2021Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigmcited by 599
  5. 2022Complexity-Based Prompting for Multi-Step Reasoningcited by 387
  6. 2021Learning How to Ask: Querying LMs with Mixtures of Soft Promptscited by 376
  7. 2023Chain-of-Thought Hub: A Continuous Effort to Measure Large Language Models' Reasoning Performancecited by 102
  8. 2024Beyond the Limits: A Survey of Techniques to Extend the Context Length in Large Language Modelscited by 71
  9. 2022Interactive and Visual Prompt Engineering for Ad-hoc Task Adaptation with Large Language Modelscited by 70
  10. 2023FELM: Benchmarking Factuality Evaluation of Large Language Modelscited by 66
  11. 2024Efficient Interactive LLM Serving with Proxy Model-based Sequence Length Predictioncited by 64
  12. 2019Positional Encoding to Control Output Sequence Lengthcited by 56
  13. 2021GooAQ: Open Question Answering with Diverse Answer Typescited by 53
  14. 2023What's the Magic Word? A Control Theory of LLM Promptingcited by 41
  15. 2024Towards Optimizing the Costs of LLM Usagecited by 33

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 65; 2026: 39.

108 papers.

Sort:

By area:

  1. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 381
  2. 2025Chain of Draft: Thinking Faster by Writing Lesscited by 290
  3. 2024Token-Budget-Aware LLM Reasoningcited by 242
  4. 2025Training Language Models to Reason Efficientlycited by 242
  5. 2025When More is Less: Understanding Chain-of-Thought Length in LLMscited by 182
  6. 2025DAST: Difficulty-Adaptive Slow-Thinking for Large Reasoning Modelscited by 154
  7. 2025A Survey of Efficient Reasoning for Large Reasoning Models: Language, Multimodality, and Beyondcited by 131
  8. 2025Sketch-of-Thought: Efficient LLM Reasoning with Adaptive Cognitive-Inspired Sketchingcited by 116
  9. 2025LightThinker: Thinking Step-by-Step Compressioncited by 104
  10. 2025Self-Training Elicits Concise Reasoning in Large Language Modelscited by 97
  11. 2025How Well do LLMs Compress Their Own Chain-of-Thought? A Token Complexity Approachcited by 95
  12. 2025Don't Overthink it. Preferring Shorter Thinking Chains for Improved LLM Reasoningcited by 82
38 more
  1. 2024INCLUDE: Evaluating Multilingual Language Understanding with Regional Knowledgecited by 78
  2. 2025Concise Reasoning via Reinforcement Learningcited by 78
  3. 2025ShorterBetter: Guiding Reasoning Models to Find Optimal Inference Length for Efficient Reasoningcited by 70
  4. 2025Efficient Inference for Large Reasoning Models: A Surveycited by 58
  5. 2025AdaCtrl: Towards Adaptive and Controllable Reasoning via Difficulty-Aware Budgetingcited by 50
  6. 2025SelfBudgeter: Adaptive Token Allocation for Efficient LLM Reasoningcited by 47
  7. 2025A Survey of State of the Art Large Vision Language Models: Alignment, Benchmark, Evaluations and Challengescited by 46
  8. 2025Dynamic Parallel Tree Search for Efficient LLM Reasoningcited by 42
  9. 2025Budget-Aware Tool Use Enables Effective Agent Scalingcited by 42
  10. 2025Answer Convergence as a Signal for Early Stopping in Reasoningcited by 38
  11. 2025Don't Overthink It: A Survey of Efficient R1-style Large Reasoning Modelscited by 36
  12. 2025Harnessing the Reasoning Economy: A Survey of Efficient Reasoning for Large Language Modelscited by 35
  13. 2025Cost-of-Pass: An Economic Framework for Evaluating Language Modelscited by 35
  14. 2025Prolonged Reasoning Is Not All You Need: Certainty-Based Adaptive Routing for Efficient LLM/MLLM Reasoningcited by 34
  15. 2025ConCISE: Confidence-guided Compression in Step-by-step Efficient Reasoningcited by 33
  16. 2025Reasoning on a Budget: A Survey of Adaptive and Controllable Test-Time Compute in LLMscited by 31
  17. 2025The Curse of CoT: On the Limitations of Chain-of-Thought in In-Context Learningcited by 30
  18. 2025Optimizing Anytime Reasoning via Budget Relative Policy Optimizationcited by 27
  19. 2025SPEED-RL: Faster Training of Reasoning Models via Online Curriculum Learningcited by 27
  20. 2025Thinking Fast and Right: Balancing Accuracy and Reasoning Length with Adaptive Rewardscited by 20
  21. 2025Towards Concise and Adaptive Thinking in Large Reasoning Models: A Surveycited by 18
  22. 2025An Empirical Study of LLM Reasoning Ability Under Strict Output Length Constraintcited by 16
  23. 2025QFFT, Question-Free Fine-Tuning for Adaptive Reasoningcited by 15
  24. 2025Generative AI Act II: Test Time Scaling Drives Cognition Engineeringcited by 14
  25. 2026R2-Router: A New Paradigm for LLM Routing with Reasoningcited by 13
  26. 2026Inference-Time Budget Control for LLM Search Agentscited by 13
  27. 2025Let Me Think! A Long Chain-of-Thought Can Be Worth Exponentially Many Short Onescited by 12
  28. 2025REST: Stress Testing Large Reasoning Models by Asking Multiple Problems at Oncecited by 12
  29. 2025AdapThink: Adaptive Thinking Preferences for Reasoning Language Modelcited by 11
  30. 2025AALC: Large Language Model Efficient Reasoning via Adaptive Accuracy-Length Controlcited by 11
  31. 2025From Efficiency to Adaptivity: A Deeper Look at Adaptive Reasoning in Large Language Modelscited by 11
  32. 2024Rethinking Chain-of-Thought from the Perspective of Self-Trainingcited by 10
  33. 2025MetaSynth: Meta-Prompting-Driven Agentic Scaffolds for Diverse Synthetic Data Generationcited by 10
  34. 2025Climplicit: Climatic Implicit Embeddings for Global Ecological Taskscited by 9
  35. 2025The Overthinker's DIET: Cutting Token Calories with DIfficulty-AwarE Trainingcited by 9
  36. 2025Verbalized Confidence Triggers Self-Verification: Emergent Behavior Without Explicit Reasoning Supervisioncited by 9
  37. 2025TeachLM: Post-Training LLMs for Education Using Authentic Learning Datacited by 8
  38. 2025Scalable LLM Reasoning Acceleration with Low-rank Distillationcited by 7

50 most cited of 108 All 108 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Understanding Tone-Dependent Inference Cost in Large Language Models
  2. 2024Same Task, More Tokens: the Impact of Input Length on the Reasoning Performance of Large Language Models
  3. 2024The Impact of Reasoning Step Length on Large Language Models
  4. 2025Thinking Short and Right Over Thinking Long: Serving LLM Reasoning Efficiently and Accurately
  5. 2025Fast on the Easy, Deep on the Hard: Efficient Reasoning via Powered Length Penalty
  6. 2026Small Language Model Helps Resolve Semantic Ambiguity of LLM Prompt
  7. 2025Towards Thinking-Optimal Scaling of Test-Time Compute for LLM Reasoning
  8. 2025Between Underthinking and Overthinking: An Empirical Study of Reasoning Length and correctness in LLMs