mlsanityData to 14 Oct 2026

CROP: Token-Efficient Reasoning in Large Language Models via Regularized Prompt Optimization

arXiv 2604.14214 ↗First posted 8 Apr 2026cs.CL cs.AIcited by 0

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

Abstract (arXiv)

Large Language Models utilizing reasoning techniques improve task performance but incur significant latency and token costs due to verbose generation. Existing automatic prompt optimization(APO) frameworks target task accuracy exclusively at the expense of generating long reasoning traces. We propose Cost-Regularized Optimization of Prompts (CROP), an APO method that introduces regularization on response length by generating textual feedback in addition to standard accuracy feedback. This forces the optimization process to produce prompts that elicit concise responses containing only critical information and reasoning. We evaluate our approach on complex reasoning datasets, specifically GSM8K, LogiQA and BIG-Bench Hard. We achieved an 80.6% reduction in token consumption while maintaining competitive accuracy, seeing only a nominal decline in performance. This presents a pragmatic solution for deploying token-efficient and cost-effective agentic AI systems in production pipelines.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 28 of its 32 references that are in the map. First posted on arXiv 8 Apr 2026.

Builds on

28 of 32 references.

Closest ancestors · 6

Same direction, newest first

  1. 2026Prompt-Level Distillation: A Non-Parametric Alternative to Model Fine-Tuning for Efficient Reasoning24 Feb
  2. 2025CODI: Compressing Chain-of-Thought into Continuous Space via Self-Distillation28 Feb
  3. 2025Chain of Draft: Thinking Faster by Writing Less25 Feb
  4. 2024Training Large Language Models to Reason in a Continuous Latent Space9 Dec
  5. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them17 Oct
  6. 2022Large Language Models are Zero-Shot Reasoners24 May

Other references · 22

  1. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  2. 2023Let's Verify Step by Stepcited by 3,789
  3. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,998
  4. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,045
  5. 2023BloombergGPT: A Large Language Model for Financecited by 939
  6. 2021Show Your Work: Scratchpads for Intermediate Computation with Language Modelscited by 886
  7. 2023FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performancecited by 853
  8. 2024RouteLLM: Learning to Route LLMs with Preference Datacited by 682
  9. 2023Automatic Prompt Optimization with "Gradient Descent" and Beam Searchcited by 653
  10. 2022DeepSpeed Inference: Enabling Efficient Inference of Transformer Models at Unprecedented Scalecited by 407
  11. 2022RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learningcited by 366
  12. 2025TokenSkip: Controllable Chain-of-Thought Compression in LLMscited by 249
10 more
  1. 2024C3oT: Generating Shorter Chain-of-Thought without Compromising Effectivenesscited by 119
  2. 2025How Well do LLMs Compress Their Own Chain-of-Thought? A Token Complexity Approachcited by 97
  3. 2023EcoAssistant: Using LLM Assistant More Affordably and Accuratelycited by 49
  4. 2025AutoPDL: Automatic Prompt Optimization for LLM Agentscited by 27
  5. 2024MOPO: Multi-Objective Prompt Optimization for Affective Text Generationcited by 9
  6. 2025An Empirical Study on Prompt Compression for Large Language Modelscited by 8
  7. 2025CAPO: Cost-Aware Prompt Optimizationcited by 7
  8. 2025metaTextGrad: Automatically optimizing language model optimizerscited by 6
  9. 2026Long-Tail Knowledge in Large Language Models: Taxonomy, Mechanisms, Interventions and Implicationscited by 5
  10. 2026Taxonomy of the Retrieval System Framework: Pitfalls and Paradigmscited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025PREMISE: Scalable and Strategic Prompt Optimization for Efficient Mathematical Reasoning in Large Models
  2. 2024GReaTer: Gradients over Reasoning Makes Smaller Language Models Strong Prompt Optimizers
  3. 2026STEVE: Stabilizing Textual Gradient-Based Prompt Optimization via Error-Driven Refinement and Regularized Verification
  4. 2023Batch Prompting: Efficient Inference with Large Language Model APIs
  5. 2026Agent-GWO: Collaborative Agents for Dynamic Prompt Optimization in Large Language Models
  6. 2025MA-SAPO: Multi-Agent Reasoning for Score-Aware Prompt Optimization
  7. 2025PromptCoT 2.0: Scaling Prompt Synthesis for Large Language Model Reasoning
  8. 2025ProRefine: Inference-Time Prompt Refinement with Textual Feedback