mlsanityData to 14 Oct 2026

Towards Optimizing with Large Language Models

arXiv 2310.05204 ↗First posted 8 Oct 2023cs.LGcited by 35

Not on the map. Why?

Abstract (arXiv)

In this work, we conduct an assessment of the optimization capabilities of LLMs across various tasks and data sizes. Each of these tasks corresponds to unique optimization domains, and LLMs are required to execute these tasks with interactive prompting. That is, in each optimization step, the LLM generates new solutions from the past generated solutions with their values, and then the new solutions are evaluated and considered in the next optimization step. Additionally, we introduce three distinct metrics for a comprehensive assessment of task performance from various perspectives. These metrics offer the advantage of being applicable for evaluating LLM performance across a broad spectrum of optimization tasks and are less sensitive to variations in test samples. By applying these metrics, we observe that LLMs exhibit strong optimization capabilities when dealing with small-sized samples. However, their performance is significantly influenced by factors like data size and values, underscoring the importance of further research in the domain of optimization tasks for LLMs.

Key facts. Cited by 35 papers in the map (data through 14 Oct 2026). Builds on 7 of its 11 references that are in the map. First posted on arXiv 8 Oct 2023.

Builds on

7 of 11 references.

References · 7

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  2. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  3. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  4. 2023Large Language Models as Optimizerscited by 864
  5. 2021Understanding Factuality in Abstractive Summarization with FRANK: A Benchmark for Factuality Metricscited by 287
  6. 2023EvoPrompt: Connecting LLMs with Evolutionary Algorithms Yields Powerful Prompt Optimizerscited by 193
  7. 2022Rich Knowledge Sources Bring Complex Knowledge Conflicts: Recalibrating Models to Reflect Conflicting Evidencecited by 144

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 16; 2025: 13; 2026: 4.

35 papers.

Sort:

By area:

  1. 2024Evolution of Heuristics: Towards Efficient Automatic Algorithm Design Using Large Language Modelcited by 268
  2. 2025Monte Carlo Tree Search for Comprehensive Exploration in LLM-Based Automatic Heuristic Designcited by 109
  3. 2024Evolutionary Computation in the Era of Large Language Model: Survey and Roadmapcited by 90
  4. 2024Large Language Model (LLM) for Telecommunications: A Comprehensive Survey on Principles, Key Techniques, and Opportunitiescited by 83
  5. 2023Algorithm Evolution Using Large Language Modelcited by 53
  6. 2024LLaMoCo: Instruction Tuning of Large Language Models for Optimization Code Generationcited by 45
  7. 2024LLMPhy: Parameter-Identifiable Physical Reasoning Combining Large Language Models and Physics Enginescited by 30
  8. 2023Large Language Model for Multi-objective Evolutionary Optimizationcited by 29
  9. 2024Large Language Model-Based Evolutionary Optimizer: Reasoning with elitismcited by 27
  10. 2024Toward Automated Algorithm Design: A Survey and Practical Guide to Meta-Black-Box-Optimizationcited by 24
  11. 2025Language-Based Bayesian Optimization Research Assistant (BORA)cited by 15
  12. 2024LLM4ED: Large Language Models for Automatic Equation Discoverycited by 14
23 more
  1. 2025LLINBO: Trustworthy LLM-in-the-Loop Bayesian Optimizationcited by 14
  2. 2024How Multimodal Integration Boost the Performance of LLM for Optimization: Case Study on Capacitated Vehicle Routing Problemscited by 12
  3. 2024Deep Insights into Automated Optimization with Large Language Models and Evolutionary Algorithmscited by 12
  4. 2025A Systematic Survey on Large Language Models for Evolutionary Optimization: From Modeling to Solvingcited by 11
  5. 2024Generalists vs. Specialists: Evaluating LLMs on Highly-Constrained Biophysical Sequence Optimization Taskscited by 10
  6. 2024Metaheuristics and Large Language Models Join Forces: Toward an Integrated Optimization Approachcited by 7
  7. 2024Using Large Language Models for Parametric Shape Optimizationcited by 7
  8. 2025MoE2: Optimizing Collaborative Inference for Edge Large Language Modelscited by 6
  9. 2025Large Language Models in Operations Research: Methods, Applications, and Challengescited by 6
  10. 2024Identify Critical Nodes in Complex Network with Large Language Modelscited by 5
  11. 2026OPT-Engine: Benchmarking the Limits of LLMs in Optimization Modeling via Complexity Scalingcited by 5
  12. 2025Uncertainty Profiles for LLMs: Uncertainty Source Decomposition and Adaptive Model-Metric Selectioncited by 4
  13. 2024Large Language Models for Tuning Evolution Strategiescited by 2
  14. 2025From Understanding to Excelling: Template-Free Algorithm Design through Structural-Functional Co-Evolutioncited by 2
  15. 2025Automating the loop in traffic incident management on highwaycited by 2
  16. 2025InnateCoder: Learning Programmatic Options with Foundation Modelscited by 2
  17. 2025Multimodal Chip Physical Design Engineer Assistantcited by 1
  18. 2025SOLID: a Framework of Synergizing Optimization and LLMs for Intelligent Decision-Makingcited by 1
  19. 2026A Component-Based Survey of Interactions between Large Language Models and Multi-Armed Banditscited by 1
  20. 2026TurboAgent: An LLM-Driven Autonomous Multi-Agent Framework for Turbomachinery Aerodynamic Designcited by 1
  21. 2024Large Language Models for Knowledge-Free Network Management: Feasibility Study and Opportunitiescited by 0
  22. 2025Reasoning with Preference Constraints: A Benchmark for Language Models in Many-to-One Matching Marketscited by 0
  23. 2026Think in Strategies, Act with Tools: Traceable Agentic Optimization for Combinatorial Problemscited by 0