# Solving General Natural-Language-Description Optimization Problems with Large Language Models

arXiv: https://arxiv.org/abs/2407.07924 · HTML: https://mlsanity.com/paper/2407.07924/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 25 references that are in the map.
- First posted on arXiv 9 Jul 2024.

## Builds on (most cited first, 18 of 18)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Parameter-Efficient Transfer Learning for NLP](https://mlsanity.com/paper/1902.00751/) (2019)
- [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023)
- [Are NLP Models really able to Solve Simple Math Word Problems?](https://mlsanity.com/paper/2103.07191/) (2021)
- [LeanDojo: Theorem Proving with Retrieval-Augmented Language Models](https://mlsanity.com/paper/2306.15626/) (2023)
- [Black-Box Tuning for Language-Model-as-a-Service](https://mlsanity.com/paper/2201.03514/) (2022)
- [Reasoning with Language Model Prompting: A Survey](https://mlsanity.com/paper/2212.09597/) (2022)
- [NL4Opt Competition: Formulating Optimization Problems Based on Their Natural Language Descriptions](https://mlsanity.com/paper/2303.08233/) (2023)
- [DeepSpeed-Chat: Easy, Fast and Affordable RLHF Training of ChatGPT-like Models at All Scales](https://mlsanity.com/paper/2308.01320/) (2023)
- [Reinforcement-Learning based Portfolio Management with Augmented Asset Movement Prediction States](https://mlsanity.com/paper/2002.05780/) (2020)
- [Augmenting Operations Research with Auto-Formulation of Optimization Models from Problem Descriptions](https://mlsanity.com/paper/2209.15565/) (2022)
- [Fine-tuning Large Enterprise Language Models via Ontological Reasoning](https://mlsanity.com/paper/2306.10723/) (2023)

## Built on by (most cited first, 22 of 22)

- [A Survey of Optimization Modeling Meets LLMs: Progress and Future Directions](https://mlsanity.com/paper/2508.10047/) (2025)
- [Large Language Models as End-to-end Combinatorial Optimization Solvers](https://mlsanity.com/paper/2509.16865/) (2025)
- [One Size doesn't Fit All: A Personalized Conversational Tutoring Agent for Mathematics Instruction](https://mlsanity.com/paper/2502.12633/) (2025)
- [ARS: Automatic Routing Solver with Large Language Models](https://mlsanity.com/paper/2502.15359/) (2025)
- [A Systematic Survey on Large Language Models for Evolutionary Optimization: From Modeling to Solving](https://mlsanity.com/paper/2509.08269/) (2025)
- [Decision Information Meets Large Language Models: The Future of Explainable Operations Research](https://mlsanity.com/paper/2502.09994/) (2025)
- [Bridging Large Language Models and Optimization: A Unified Framework for Text-attributed Combinatorial Optimization](https://mlsanity.com/paper/2408.12214/) (2024)
- [SAC-Opt: Semantic Anchors for Iterative Correction in Optimization Modeling](https://mlsanity.com/paper/2510.05115/) (2025)
- [Large Language Models in Operations Research: Methods, Applications, and Challenges](https://mlsanity.com/paper/2509.18180/) (2025)
- [Abstract Operations Research Modeling Using Natural Language Inputs](https://mlsanity.com/paper/2408.07272/) (2024)
- [Satellite Edge Artificial Intelligence with Large Models: Architectures and Technologies](https://mlsanity.com/paper/2504.01676/) (2025)
- [CHORUS: Zero-shot Hierarchical Retrieval and Orchestration for Generating Linear Programming Code](https://mlsanity.com/paper/2505.01485/) (2025)
- [ORPilot: A Production-Oriented Agentic LLM-for-OR Tool for Optimization Modeling](https://mlsanity.com/paper/2605.02728/) (2026)
- [Intelligent Channel Allocation for IEEE 802.11be Multi-Link Operation: When MAB Meets LLM](https://mlsanity.com/paper/2506.04594/) (2025)
- [Agentic MIP Research: Accelerated Constraint Handler Generation](https://mlsanity.com/paper/2605.09186/) (2026)
- [Automated Reformulation of Robust Optimization via Memory-Augmented Large Language Models](https://mlsanity.com/paper/2605.11813/) (2026)
- [PEARL: Solver-in-the-Loop Interactive Optimization Modeling from Natural Language](https://mlsanity.com/paper/2607.18256/) (2026)
- [Reinforcement Learning Enhanced LLM Agents for Complex Vehicle Routing Problems](https://mlsanity.com/paper/2609.00859/) (2026)
- [Large Language Models as Amortized Pareto-Front Generators for Constrained Bi-Objective Convex Optimization](https://mlsanity.com/paper/2605.12106/) (2026)
- [Generating Robust Portfolios of Optimization Models using Large Language Models](https://mlsanity.com/paper/2605.27013/) (2026)
- [LLMs as Adaptive Meta-Solvers: Strategy-Diverse RL for Industrial-Scale Optimization](https://mlsanity.com/paper/2609.34427/) (2026)
- [TACIT: Optimization Models that Learn from Their Mistakes](https://mlsanity.com/paper/2609.38434/) (2026)

## Abstract (arXiv)

Optimization problems seek to find the best solution to an objective under a set of constraints, and have been widely investigated in real-world applications. Modeling and solving optimization problems in a specific domain typically require a combination of domain knowledge, mathematical skills, and programming ability, making it difficult for general users and even domain professionals. In this paper, we propose a novel framework called OptLLM that augments LLMs with external solvers. Specifically, OptLLM accepts user queries in natural language, convert them into mathematical formulations and programming codes, and calls the solvers to calculate the results for decision-making. In addition, OptLLM supports multi-round dialogues to gradually refine the modeling and solving of optimization problems. To illustrate the effectiveness of OptLLM, we provide tutorials on three typical optimization applications and conduct experiments on both prompt-based GPT models and a fine-tuned Qwen model using a large-scale selfdeveloped optimization dataset. Experimental results show that OptLLM works with various LLMs, and the fine-tuned model achieves an accuracy boost compared to the promptbased models. Some features of OptLLM framework have been available for trial since June 2023 (https://opt.alibabacloud.com/chat or https://opt.aliyun.com/chat).