# Autoformulation of Mathematical Optimization Models Using LLMs

arXiv: https://arxiv.org/abs/2411.01679 · HTML: https://mlsanity.com/paper/2411.01679/

## Key facts

- Cited by 48 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 59 references that are in the map.
- First posted on arXiv 3 Nov 2024.

## Builds on (most cited first, 29 of 29)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Let's Verify Step by Step](https://mlsanity.com/paper/2305.20050/) (2023)
- [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023)
- [Tree of Thoughts: Deliberate Problem Solving with Large Language Models](https://mlsanity.com/paper/2305.10601/) (2023)
- [Convex optimization](https://mlsanity.com/paper/2106.01946/) (2021)
- [Solving Quantitative Reasoning Problems with Language Models](https://mlsanity.com/paper/2206.14858/) (2022)
- [Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents](https://mlsanity.com/paper/2201.07207/) (2022)
- [Large Language Models as Optimizers](https://mlsanity.com/paper/2309.03409/) (2023)
- [Reasoning with Language Model is Planning with World Model](https://mlsanity.com/paper/2305.14992/) (2023)
- [MINE: Mutual Information Neural Estimation](https://mlsanity.com/paper/1801.04062/) (2018)
- [Self-Rewarding Language Models](https://mlsanity.com/paper/2401.10020/) (2024)
- [CVXPY: A Python-Embedded Modeling Language for Convex Optimization](https://mlsanity.com/paper/1603.00943/) (2016)
- [Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Models](https://mlsanity.com/paper/2310.04406/) (2023)
- [Cognitive Architectures for Language Agents](https://mlsanity.com/paper/2309.02427/) (2023)
- [Understanding the planning of LLM agents: A survey](https://mlsanity.com/paper/2402.02716/) (2024)
- [Alphazero-like Tree-Search can Guide Large Language Model Decoding and Training](https://mlsanity.com/paper/2309.17179/) (2023)
- [Large Language Models as Commonsense Knowledge for Large-Scale Task Planning](https://mlsanity.com/paper/2305.14078/) (2023)
- [Accessing GPT-4 level Mathematical Olympiad Solutions via Monte Carlo Tree Self-refine with LLaMa-3 8B](https://mlsanity.com/paper/2406.07394/) (2024)
- [NL4Opt Competition: Formulating Optimization Problems Based on Their Natural Language Descriptions](https://mlsanity.com/paper/2303.08233/) (2023)
- [Large Language Models to Enhance Bayesian Optimization](https://mlsanity.com/paper/2402.03921/) (2024)
- [Large Language Models as Evolutionary Optimizers](https://mlsanity.com/paper/2310.19046/) (2023)
- [OptiMUS: Scalable Optimization Modeling with (MI)LP Solvers and Large Language Models](https://mlsanity.com/paper/2402.10172/) (2024)
- [A Survey of Satisfiability Modulo Theory](https://mlsanity.com/paper/1606.04786/) (2016)

## Built on by (most cited first, 30 of 48)

- [OptMATH: A Scalable Bidirectional Data Synthesis Framework for Optimization Modeling](https://mlsanity.com/paper/2502.11102/) (2025)
- [Solver-Informed RL: Grounding Large Language Models for Authentic Optimization Modeling](https://mlsanity.com/paper/2505.11792/) (2025)
- [A Survey of Optimization Modeling Meets LLMs: Progress and Future Directions](https://mlsanity.com/paper/2508.10047/) (2025)
- [StepORLM: A Self-Evolving Framework With Generative Process Supervision For Operations Research Language Models](https://mlsanity.com/paper/2509.22558/) (2025)
- [AlphaOPT: Formulating Optimization Programs with Self-Improving LLM Experience Library](https://mlsanity.com/paper/2510.18428/) (2025)
- [Automated Optimization Modeling via a Localizable Error-Driven Perspective](https://mlsanity.com/paper/2602.11164/) (2026)
- [A Systematic Survey on Large Language Models for Evolutionary Optimization: From Modeling to Solving](https://mlsanity.com/paper/2509.08269/) (2025)
- [OptiMind: Teaching LLMs to Think Like Optimization Experts](https://mlsanity.com/paper/2509.22979/) (2025)
- [SolverLLM: Leveraging Test-Time Scaling for Optimization Problem via LLM-Guided Search](https://mlsanity.com/paper/2510.16916/) (2025)
- [Can We Predict Before Executing Machine Learning Agents?](https://mlsanity.com/paper/2601.05930/) (2026)
- [EquivaMap: Leveraging LLMs for Automatic Equivalence Checking of Optimization Formulations](https://mlsanity.com/paper/2502.14760/) (2025)
- [LLMs for Cold-Start Cutting Plane Separator Configuration](https://mlsanity.com/paper/2412.12038/) (2024)
- [OptiChat: Bridging Optimization Models and Practitioners with Large Language Models](https://mlsanity.com/paper/2501.08406/) (2025)
- [SAC-Opt: Semantic Anchors for Iterative Correction in Optimization Modeling](https://mlsanity.com/paper/2510.05115/) (2025)
- [Large Language Models in Operations Research: Methods, Applications, and Challenges](https://mlsanity.com/paper/2509.18180/) (2025)
- [Toward a Trustworthy Optimization Modeling Agent via Verifiable Synthetic Data Generation](https://mlsanity.com/paper/2508.03117/) (2025)
- [Opt-Verifier: Unleashing the Power of LLMs for Optimization Modeling via Dual-Side Verification](https://mlsanity.com/paper/2605.29556/) (2026)
- [Performance of LLMs on Stochastic Modeling Operations Research Problems: From Theory to Practice](https://mlsanity.com/paper/2506.23924/) (2025)
- [Let's Have a Conversation: Designing and Evaluating LLM Agents for Interactive Optimization](https://mlsanity.com/paper/2604.02666/) (2026)
- [OptiVerse: A Comprehensive Benchmark towards Optimization Problem Solving](https://mlsanity.com/paper/2604.21510/) (2026)
- [Execution-Verified Reinforcement Learning for Optimization Modeling](https://mlsanity.com/paper/2604.00442/) (2026)
- [ORPilot: A Production-Oriented Agentic LLM-for-OR Tool for Optimization Modeling](https://mlsanity.com/paper/2605.02728/) (2026)
- [TriVAL: A Tri-Validation Framework for Faithful Automatic Optimization Modeling](https://mlsanity.com/paper/2605.23966/) (2026)
- [Teaching LLMs to Think Mathematically: A Critical Study of Decision-Making via Optimization](https://mlsanity.com/paper/2508.18091/) (2025)
- [AutoOR: Scalably Post-training LLMs to Autoformulate Operations Research Problems](https://mlsanity.com/paper/2604.16804/) (2026)
- [EngiAgent: Fully Connected Coordination of LLM Agents for Solving Open-ended Engineering Problems with Feasible Solutions](https://mlsanity.com/paper/2605.02289/) (2026)
- [Automated Reformulation of Robust Optimization via Memory-Augmented Large Language Models](https://mlsanity.com/paper/2605.11813/) (2026)
- [Memory-Guided Tree Search with Cross-Branch Knowledge Transfer for LLM Solver Synthesis](https://mlsanity.com/paper/2605.17539/) (2026)
- [StarOR: Synergizing Tree Search and Test-Time Reinforcement Learning for Optimization Modeling](https://mlsanity.com/paper/2606.15197/) (2026)
- [PEARL: Solver-in-the-Loop Interactive Optimization Modeling from Natural Language](https://mlsanity.com/paper/2607.18256/) (2026)

## Abstract (arXiv)

Mathematical optimization is fundamental to decision-making across diverse domains, from operations research to healthcare. Yet, translating real-world problems into optimization models remains a difficult task, often demanding specialized expertise. This paper approaches the problem of textitautoformulation: the automated creation of solver-ready optimization models from natural language problem descriptions. We identify three core challenges of autoformulation: textit(1) the vast, problem-dependent hypothesis space, textit(2) efficient and diverse exploration of this space under uncertainty, and textit(3) evaluation of formulation correctness against problem description. To address these challenges, we present a novel method leveraging textitLarge Language Models (LLMs) with textitMonte-Carlo Tree Search, exploiting the hierarchical nature of optimization modeling to generate and systematically explore possible formulations. To enhance search efficiency, we introduce symbolic pruning to eliminate trivially equivalent search paths (branches), and employ LLM-based evaluation of partial formulations to guide search. Empirical analysis on linear and mixed-integer programming benchmarks demonstrates our method's effectiveness, with significant performance gains from both LLM-based value estimation and symbolic pruning techniques.