mlsanityData to 14 Oct 2026

Optimization Problem Solving Can Transition to Evolutionary Agentic Workflows

arXiv 2505.04354 ↗First posted 7 May 2025math.OC cs.AIcited by 0

On the map: LLM agents and tool use → Software engineering coding agents

Abstract (arXiv)

This position paper argues that optimization problem solving can transition from expert-dependent to evolutionary agentic workflows. Traditional optimization practices rely on human specialists for problem formulation, algorithm selection, and hyperparameter tuning, creating bottlenecks that impede industrial adoption of cutting-edge methods. We contend that an evolutionary agentic workflow, powered by foundation models and evolutionary search, can autonomously navigate the optimization space, comprising problem, formulation, algorithm, and hyperparameter spaces. Through case studies in cloud resource scheduling and ADMM parameter adaptation, we demonstrate how this approach can bridge the gap between academic innovation and industrial implementation. Our position challenges the status quo of human-centric optimization workflows and advocates for a more scalable, adaptive approach to solving real-world optimization problems.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 101 of its 173 references that are in the map. First posted on arXiv 7 May 2025.

Builds on

101 of 173 references.

References · 101

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  4. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  5. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  6. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  7. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  8. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,568
  9. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  10. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
  11. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  12. 2021Convex optimizationcited by 2,515
38 more
  1. 2019Optuna: A Next-generation Hyperparameter Optimization Frameworkcited by 2,501
  2. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,191
  3. 2023Code Llama: Open Foundation Models for Codecited by 2,139
  4. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,439
  5. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,298
  6. 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,190
  7. 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,081
  8. 2016An overview of gradient descent optimization algorithmscited by 1,033
  9. 2020Meta-Learning in Neural Networks: A Surveycited by 950
  10. 2018Regularized Evolution for Image Classifier Architecture Searchcited by 909
  11. 2023Large Language Models as Optimizerscited by 864
  12. 2023Reasoning with Language Model is Planning with World Modelcited by 861
  13. 2018Efficient Neural Architecture Search via Parameter Sharingcited by 837
  14. 2023ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debatecited by 742
  15. 2020It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learnerscited by 669
  16. 2024A Survey on Large Language Models for Code Generationcited by 658
  17. 2016Learning to learn by gradient descent by gradient descentcited by 620
  18. 2023Eureka: Human-Level Reward Design via Coding Large Language Modelscited by 571
  19. 2022Decomposed Prompting: A Modular Approach for Solving Complex Taskscited by 558
  20. 2024GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Modelscited by 550
  21. 2018Bilevel Programming for Hyperparameter Optimization and Meta-Learningcited by 405
  22. 2025From System 1 to System 2: A Survey of Reasoning Large Language Modelscited by 320
  23. 2025Towards Large Reasoning Models: A Survey of Reinforced Reasoning with Large Language Modelscited by 293
  24. 2024A Survey on Knowledge Distillation of Large Language Modelscited by 284
  25. 2024ReEvo: Large Language Models as Hyper-Heuristics with Reflective Evolutioncited by 276
  26. 2024Evolution of Heuristics: Towards Efficient Automatic Algorithm Design Using Large Language Modelcited by 268
  27. 2023FireAct: Toward Language Agent Fine-tuningcited by 256
  28. 2024Data Interpreter: An LLM Agent For Data Sciencecited by 225
  29. 2021Learning to Optimize: A Primer and A Benchmarkcited by 208
  30. 2023OpenAGI: When LLM Meets Domain Expertscited by 197
  31. 2023Navigate through Enigmatic Labyrinth A Survey of Chain of Thought Reasoning: Advances, Frontiers and Futurecited by 184
  32. 2020Auto-Sklearn 2.0: Hands-free AutoML via Meta-Learningcited by 174
  33. 2023NL4Opt Competition: Formulating Optimization Problems Based on Their Natural Language Descriptionscited by 135
  34. 2024Exploring Large Language Model based Intelligent Agents: Definitions, Methods, and Prospectscited by 132
  35. 2023Language Model Crossover: Variation through Few-Shot Promptingcited by 122
  36. 2024To Believe or Not to Believe Your LLMcited by 120
  37. 2024OptiMUS: Scalable Optimization Modeling with (MI)LP Solvers and Large Language Modelscited by 109
  38. 2018Understanding and correcting pathologies in the training of learned optimizerscited by 97

50 most cited of 101 All 101 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Co-evolving Agent Architectures and Interpretable Reasoning for Automated Optimization
  2. 2026OR-Agent: Bridging Evolutionary Search and Structured Research for Automated Heuristic Design
  3. 2025EvoFlow: Evolving Diverse Agentic Workflows On The Fly
  4. 2025MermaidFlow: Redefining Agentic Workflow Generation via Safety-Constrained Evolutionary Programming
  5. 2026HeurEvo: Agentic Evolution of Hybrid Solver-Augmented Heuristics for Time-Critical Mathematical Optimization
  6. 2025Orchestrating Human-AI Teams: The Manager Agent as a Unifying Research Challenge
  7. 2025The (R)evolution of Scientific Workflows in the Agentic AI Era: Towards Autonomous Science
  8. 2026Fusion is the New Mutation: Bandit-Guided Evolution on Workflow Graphs