# Constructing Industrial-Scale Optimization Modeling Benchmark

arXiv: https://arxiv.org/abs/2602.10450 · HTML: https://mlsanity.com/paper/2602.10450/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 25 references that are in the map.
- First posted on arXiv 11 Feb 2026.

## Builds on (most cited first, 14 of 14)

- [NL4Opt Competition: Formulating Optimization Problems Based on Their Natural Language Descriptions](https://mlsanity.com/paper/2303.08233/) (2023)
- [ORLM: A Customizable Framework in Training Large Models for Automated Optimization Modeling](https://mlsanity.com/paper/2405.17743/) (2024)
- [OptiMUS: Scalable Optimization Modeling with (MI)LP Solvers and Large Language Models](https://mlsanity.com/paper/2402.10172/) (2024)
- [LLMOPT: Learning to Define and Solve General Optimization Problems from Scratch](https://mlsanity.com/paper/2410.13213/) (2024)
- [OptiBench Meets ReSocratic: Measure and Improve LLMs for Optimization Modeling](https://mlsanity.com/paper/2407.09887/) (2024)
- [LLMs for Mathematical Modeling: Towards Bridging the Gap between Natural and Mathematical Languages](https://mlsanity.com/paper/2405.13144/) (2024)
- [OptMATH: A Scalable Bidirectional Data Synthesis Framework for Optimization Modeling](https://mlsanity.com/paper/2502.11102/) (2025)
- [Solver-Informed RL: Grounding Large Language Models for Authentic Optimization Modeling](https://mlsanity.com/paper/2505.11792/) (2025)
- [A Survey of Optimization Modeling Meets LLMs: Progress and Future Directions](https://mlsanity.com/paper/2508.10047/) (2025)
- [Learning to Handle Complex Constraints for Vehicle Routing Problems](https://mlsanity.com/paper/2410.21066/) (2024)
- [Synthesizing mixed-integer linear programming models from natural language descriptions](https://mlsanity.com/paper/2311.15271/) (2023)
- [Automated Optimization Modeling via a Localizable Error-Driven Perspective](https://mlsanity.com/paper/2602.11164/) (2026)
- [ORGEval: Graph-Theoretic Evaluation of LLMs in Optimization Modeling](https://mlsanity.com/paper/2510.27610/) (2025)
- [Toward a Trustworthy Optimization Modeling Agent via Verifiable Synthetic Data Generation](https://mlsanity.com/paper/2508.03117/) (2025)

## Built on by (most cited first, 12 of 12)

- [OptArgus: A Multi-Agent System to Detect Hallucinations in LLM-based Optimization Modeling](https://mlsanity.com/paper/2605.11738/) (2026)
- [MM-OptBench: A Solver-Grounded Benchmark for Multimodal Optimization Modeling](https://mlsanity.com/paper/2605.12154/) (2026)
- [OR-Space: A Full-Lifecycle Workspace Benchmark for Industrial Optimization Agents](https://mlsanity.com/paper/2605.28158/) (2026)
- [OptSkills: Learning Generalizable Optimization Skills from Problem Archetypes via Cluster-Based Distillation](https://mlsanity.com/paper/2605.29829/) (2026)
- [Language Models as Higher-Order Planning Formalizers](https://mlsanity.com/paper/2603.23844/) (2026)
- [ORAgentBench: Can LLM Agents Solve Challenging Operations Research Tasks End to End?](https://mlsanity.com/paper/2606.19787/) (2026)
- [EvoOptiGraph: Weakness-Driven Coevolution via Graph-Based Structural Generation for Optimization Modeling](https://mlsanity.com/paper/2606.26578/) (2026)
- [Learning to Optimize through Solver-Grounded Self-Play](https://mlsanity.com/paper/2609.34205/) (2026)
- [LLMs as Adaptive Meta-Solvers: Strategy-Diverse RL for Industrial-Scale Optimization](https://mlsanity.com/paper/2609.34427/) (2026)
- [HeurEvo: Agentic Evolution of Hybrid Solver-Augmented Heuristics for Time-Critical Mathematical Optimization](https://mlsanity.com/paper/2609.36303/) (2026)
- [Right Answers, Costly Models: The Efficiency Gap in LLM-based Optimization Modeling](https://mlsanity.com/paper/2609.38884/) (2026)
- [FDE-Bench: Evaluating End-to-End Delivery from Underspecified Real-World Business Requests](https://mlsanity.com/paper/2610.14539/) (2026)

## Abstract (arXiv)

Optimization modeling underpins decision-making in logistics, manufacturing, energy, and finance, yet translating natural-language requirements into correct optimization formulations and solver-executable code remains labor-intensive. Although large language models (LLMs) have been explored for this task, evaluation is still dominated by toy-sized or synthetic benchmarks, masking the difficulty of industrial problems with 103--106 (or more) variables and constraints. A key bottleneck is the lack of benchmarks that align natural-language specifications with reference formulations/solver code grounded in real optimization models. To fill in this gap, we introduce MIPLIB-NL, built via a structure-aware reverse construction methodology from real mixed-integer linear programs in MIPLIB~2017. Our pipeline (i) recovers compact, reusable model structure from flat solver formulations, (ii) reverse-generates natural-language specifications explicitly tied to this recovered structure under a unified model--data separation format, and (iii) performs iterative semantic validation through expert review and human--LLM interaction with independent reconstruction checks. This yields 223 one-to-one reconstructions that preserve the mathematical content of the original instances while enabling realistic natural-language-to-optimization evaluation. Experiments show substantial performance degradation on MIPLIB-NL for systems that perform strongly on existing benchmarks, exposing failure modes invisible at toy scale.