mlsanityData to 7 Oct 2026

Cost-Effective Hyperparameter Optimization for Large Language Model Generation Inference

arXiv 2303.04673 ↗First posted 8 Mar 2023cs.CL cs.AI cs.LGcited by 40

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) have sparked significant interest in their generative capabilities, leading to the development of various commercial applications. The high cost of using the models drives application builders to maximize the value of generation under a limited inference budget. This paper presents a study of optimizing inference hyperparameters such as the number of responses, temperature and max tokens, which significantly affects the utility/cost of text generation. We design a framework named EcoOptiGen which leverages economical hyperparameter optimization and cost-based pruning. Experiments with the GPT-3.5/GPT-4 models on a variety of tasks verify its effectiveness. EcoOptiGen is implemented in the `autogen' package of the FLAML library: urlhttps://aka.ms/autogen.

Key facts. Cited by 40 papers in the map (data through 7 Oct 2026). Builds on 17 of its 40 references that are in the map. First posted on arXiv 8 Mar 2023.

Builds on

17 of 40 references.

References · 17

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  5. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  6. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  7. 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,308
  8. 2021Measuring Coding Challenge Competence With APPScited by 879
  9. 2022Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Taskscited by 858
  10. 2016Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimizationcited by 688
  11. 2019Green AIcited by 620
  12. 2021Reframing Instructional Prompts to GPTk's Languagecited by 176
5 more
  1. 2022Synchromesh: Reliable code generation from pre-trained language modelscited by 156
  2. 2020A Systematic Characterization of Sampling Algorithms for Open-ended Language Generationcited by 28
  3. 2021An Empirical Study on Hyperparameter Optimization for Fine-Tuning Pre-trained Language Modelscited by 12
  4. 2022Leveraging Natural Supervision for Language Representation Learning and Generationcited by 2
  5. 2022Don't Be So Sure! Boosting ASR Decoding via Confidence Relaxationcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 4; 2024: 22; 2025: 9; 2026: 5.

40 papers.

Sort:

By area:

  1. 2023PandaLM: An Automatic Evaluation Benchmark for LLM Instruction Tuning Optimizationcited by 311
  2. 2024The Effect of Sampling Temperature on Problem Solving in Large Language Modelscited by 168
  3. 2024Non-Determinism of "Deterministic" LLM Settingscited by 84
  4. 2023EcoAssistant: Using LLM Assistant More Affordably and Accuratelycited by 47
  5. 2023AutoML in the Age of Large Language Models: Current Challenges, Future Opportunities and Riskscited by 34
  6. 2024LLMParser: An Exploratory Study on Using Large Language Models for Log Parsingcited by 32
  7. 2024How Far Are We From AGI: Are LLMs All We Need?cited by 31
  8. 2024Large Language Models as Test Case Generators: Performance Evaluation and Enhancementcited by 27
  9. 2025ReCode: Reinforcing Code Generation with Reasoning-Process Rewardscited by 25
  10. 2024FunBO: Discovering Acquisition Functions for Bayesian Optimization with FunSearchcited by 22
  11. 2025Optimizing the Interface Between Knowledge Graphs and LLMs for Complex Reasoningcited by 17
  12. 2024Scaling LLM Inference with Optimized Sample Compute Allocationcited by 16
28 more
  1. 2024CEval: A Benchmark for Evaluating Counterfactual Text Generationcited by 15
  2. 2024CigaR: Cost-efficient Program Repair with LLMscited by 14
  3. 2024AutoRAG-HP: Automatic Online Hyper-Parameter Tuning for Retrieval-Augmented Generationcited by 13
  4. 2024Cognitive Overload Attack:Prompt Injection for Long Contextcited by 13
  5. 2024Problem Solving Through Human-AI Preference-Based Cooperationcited by 11
  6. 2024EcoAct: Economic Agent Determines When to Register What Actioncited by 11
  7. 2025Faster, Cheaper, Better: Multi-Objective Hyperparameter Optimization for LLM and RAG Systemscited by 11
  8. 2025MS-BART: Unified Modeling of Mass Spectra and Molecules for Structure Elucidationcited by 9
  9. 2026AutoSaddler: Automatic Harness Optimization with Durable Updates from Agent Execution Tracescited by 9
  10. 2023How Many Validation Labels Do You Need? Exploring the Design Space of Label-Efficient Model Rankingcited by 8
  11. 2024The Impact of Hyperparameters on Large Language Model Inference Performance: An Evaluation of vLLM and HuggingFace Pipelinescited by 5
  12. 2025Scaling Textual Gradients via Sampling-Based Momentumcited by 5
  13. 2024LLM can Achieve Self-Regulation via Hyperparameter Aware Generationcited by 4
  14. 2024Guiding Large Language Models to Generate Computer-Parsable Contentcited by 4
  15. 2025HyperSHAP: Shapley Values and Interactions for Explaining Hyperparameter Optimizationcited by 4
  16. 2024Language Models as Efficient Reward Function Searchers for Custom-Environment Multi-Objective Reinforcementcited by 3
  17. 2026Models Under SCOPE: Scalable and Controllable Routing via Pre-hoc Reasoningcited by 3
  18. 2024Return of EM: Entity-driven Answer Set Expansion for QA Evaluationcited by 2
  19. 2024SuperCode: Sustainability PER AI-driven CO-DEsigncited by 2
  20. 2024Hallucinations or Attention Misdirection? The Path to Strategic Value Extraction in Business Using Large Language Modelscited by 1
  21. 2024From Text to Pose to Image: Improving Diffusion Model Control and Qualitycited by 1
  22. 2025A statistically consistent measure of semantic uncertainty using Language Modelscited by 1
  23. 2025ParaVul: A Parallel Large Language Model and Retrieval-Augmented Framework for Smart Contract Vulnerability Detectioncited by 1
  24. 2026Rethinking Automated Program Repair: The Impact of Bug Complexity, Fault Localization, and LLM Cost-efficiencycited by 1
  25. 2024Synthesize, Partition, then Adapt: Eliciting Diverse Samples from Foundation Modelscited by 0
  26. 2025XAutoLM: Efficient Fine-Tuning of Language Models via Meta-Learning and AutoMLcited by 0
  27. 2026LLM-Guided Runtime Parameter Optimization for Energy-Efficient Model Inferencecited by 0
  28. 2026Persona Non Grata: LLM Persona-Driven Generations in MCQA are Unstable in Distinct Dimensionscited by 0