# OptiMUS-0.3: Using Large Language Models to Model and Solve Optimization Problems at Scale

arXiv: https://arxiv.org/abs/2407.19633 · HTML: https://mlsanity.com/paper/2407.19633/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 51 references that are in the map.
- First posted on arXiv 29 Jul 2024.

## Builds on (most cited first, 20 of 20)

- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Convex optimization](https://mlsanity.com/paper/2106.01946/) (2021)
- [Retrieval-Augmented Generation for Large Language Models: A Survey](https://mlsanity.com/paper/2312.10997/) (2023)
- [Improving language models by retrieving from trillions of tokens](https://mlsanity.com/paper/2112.04426/) (2021)
- [Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMs](https://mlsanity.com/paper/2306.13063/) (2023)
- [PAL: Program-aided Language Models](https://mlsanity.com/paper/2211.10435/) (2022)
- [CVXPY: A Python-Embedded Modeling Language for Convex Optimization](https://mlsanity.com/paper/1603.00943/) (2016)
- [Proving Test Set Contamination in Black Box Language Models](https://mlsanity.com/paper/2310.17623/) (2023)
- [ART: Automatic multi-step reasoning and tool-use for large language models](https://mlsanity.com/paper/2303.09014/) (2023)
- [Same Task, More Tokens: the Impact of Input Length on the Reasoning Performance of Large Language Models](https://mlsanity.com/paper/2402.14848/) (2024)
- [NL4Opt Competition: Formulating Optimization Problems Based on Their Natural Language Descriptions](https://mlsanity.com/paper/2303.08233/) (2023)
- [OptiMUS: Scalable Optimization Modeling with (MI)LP Solvers and Large Language Models](https://mlsanity.com/paper/2402.10172/) (2024)
- [Retrieval-Augmented Generation for Code Summarization via Hybrid GNN](https://mlsanity.com/paper/2006.05405/) (2020)
- [OptiBench Meets ReSocratic: Measure and Improve LLMs for Optimization Modeling](https://mlsanity.com/paper/2407.09887/) (2024)
- [Large Language Models for Supply Chain Optimization](https://mlsanity.com/paper/2307.03875/) (2023)
- [Augmenting Operations Research with Auto-Formulation of Optimization Models from Problem Descriptions](https://mlsanity.com/paper/2209.15565/) (2022)
- [Diagnosing Infeasible Optimization Problems Using Large Language Models](https://mlsanity.com/paper/2308.12923/) (2023)
- ["I Want It That Way": Enabling Interactive Decision Support Using Large Language Models and Constraint Programming](https://mlsanity.com/paper/2312.06908/) (2023)
- [From Large Language Models and Optimization to Decision Optimization CoPilot: A Research Manifesto](https://mlsanity.com/paper/2402.16269/) (2024)

## Built on by (most cited first, 30 of 38)

- [StepORLM: A Self-Evolving Framework With Generative Process Supervision For Operations Research Language Models](https://mlsanity.com/paper/2509.22558/) (2025)
- [OptimAI: Optimization from Natural Language Using LLM-Powered AI Agents](https://mlsanity.com/paper/2504.16918/) (2025)
- [AlphaOPT: Formulating Optimization Programs with Self-Improving LLM Experience Library](https://mlsanity.com/paper/2510.18428/) (2025)
- [Improving Parallel Program Performance with LLM Optimizers via Agent-System Interfaces](https://mlsanity.com/paper/2410.15625/) (2024)
- [A Systematic Survey on Large Language Models for Evolutionary Optimization: From Modeling to Solving](https://mlsanity.com/paper/2509.08269/) (2025)
- [LLMs for Cold-Start Cutting Plane Separator Configuration](https://mlsanity.com/paper/2412.12038/) (2024)
- [OptiChat: Bridging Optimization Models and Practitioners with Large Language Models](https://mlsanity.com/paper/2501.08406/) (2025)
- [SAC-Opt: Semantic Anchors for Iterative Correction in Optimization Modeling](https://mlsanity.com/paper/2510.05115/) (2025)
- [Text2Zinc: A Cross-Domain Dataset for Modeling Optimization and Satisfaction Problems in MiniZinc](https://mlsanity.com/paper/2503.10642/) (2025)
- [ConstraintBench: Benchmarking LLM Constraint Reasoning on Direct Optimization](https://mlsanity.com/paper/2602.22465/) (2026)
- [Large Language Models in Operations Research: Methods, Applications, and Challenges](https://mlsanity.com/paper/2509.18180/) (2025)
- [ReflecSched: Solving Dynamic Flexible Job-Shop Scheduling via LLM-Powered Hierarchical Reflection](https://mlsanity.com/paper/2508.01724/) (2025)
- [Ask, Clarify, Optimize: Human-LLM Agent Collaboration for Smarter Inventory Control](https://mlsanity.com/paper/2601.00121/) (2025)
- [NEMO: Execution-Aware Optimization Modeling via Autonomous Coding Agents](https://mlsanity.com/paper/2601.21372/) (2026)
- [DualSchool: How Reliable are LLMs for Optimization Education?](https://mlsanity.com/paper/2505.21775/) (2025)
- [Generative Reliability-Based Design Optimization Using In-Context Learning Capabilities of Large Language Models](https://mlsanity.com/paper/2503.22401/) (2025)
- [Performance of LLMs on Stochastic Modeling Operations Research Problems: From Theory to Practice](https://mlsanity.com/paper/2506.23924/) (2025)
- [From Natural Language to Solver-Ready Power System Optimization: An LLM-Assisted, Validation-in-the-Loop Framework](https://mlsanity.com/paper/2508.08147/) (2025)
- [Let's Have a Conversation: Designing and Evaluating LLM Agents for Interactive Optimization](https://mlsanity.com/paper/2604.02666/) (2026)
- [From Soliloquy to Agora: Memory-Enhanced LLM Agents with Decentralized Debate for Optimization Modeling](https://mlsanity.com/paper/2604.25847/) (2026)
- [ORPilot: A Production-Oriented Agentic LLM-for-OR Tool for Optimization Modeling](https://mlsanity.com/paper/2605.02728/) (2026)
- [OptiHive: Ensemble Selection for LLM-Based Optimization via Statistical Modeling](https://mlsanity.com/paper/2508.02503/) (2025)
- [Large Language Model-Based Automatic Formulation for Stochastic Optimization Models](https://mlsanity.com/paper/2508.17200/) (2025)
- [Teaching LLMs to Think Mathematically: A Critical Study of Decision-Making via Optimization](https://mlsanity.com/paper/2508.18091/) (2025)
- [Agentic MIP Research: Accelerated Constraint Handler Generation](https://mlsanity.com/paper/2605.09186/) (2026)
- [FLARE: Verifying MILP Reformulations with LLM-Based Theorem Proving](https://mlsanity.com/paper/2608.25220/) (2026)
- [A-LAMP: Agentic LLM-Based Framework for Automated MDP Modeling and Policy Generation](https://mlsanity.com/paper/2512.11270/) (2025)
- [Infeasibility Aware Large Language Models for Combinatorial Optimization](https://mlsanity.com/paper/2604.01455/) (2026)
- [Text2Model: Modeling Copilots for Text-to-Model Translation](https://mlsanity.com/paper/2604.12955/) (2026)
- [Co-evolving Agent Architectures and Interpretable Reasoning for Automated Optimization](https://mlsanity.com/paper/2604.17708/) (2026)

## Abstract (arXiv)

Optimization problems are pervasive in sectors from manufacturing and distribution to healthcare. However, most such problems are still solved heuristically by hand rather than optimally by state-of-the-art solvers because the expertise required to formulate and solve these problems limits the widespread adoption of optimization tools and techniques. We introduce a Large Language Model (LLM)-based system designed to formulate and solve (mixed integer) linear programming problems from their natural language descriptions. Our system can develop mathematical models, write and debug solver code, evaluate the generated solutions, and improve efficiency and correctness of its model and code based on these evaluations. OptiMUS is designed as a productivity tool for optimization practitioners who understand the problem domain and can describe it precisely, but seek to accelerate the modeling and implementation workflow. OptiMUS-0.3 utilizes a modular structure to process problems, allowing it to handle problems with long descriptions and complex data without long prompts. Experiments demonstrate that OptiMUS-0.3 outperforms direct-prompting baselines by over 43% on easy and 18% on hard instances. It remains competitive with fine-tuned specialist models on benchmark problems, and outperforms them on real-world case studies (28.6% vs. 0%) where fine-tuned models fail to generalize. Ablation studies show that modular architecture with error correction is central to these gains. A key finding is that system architecture is a stronger driver of performance than model capability. Structured decomposition with targeted error correction enables weaker models to match stronger models under naive prompting, and remains competitive with fine-tuned specialist models without retraining costs.