# Opti-Agent-Bench: Benchmarking End-to-End Optimization R&D Agents on Real-World Business Problems

arXiv: https://arxiv.org/abs/2607.10768 · HTML: https://mlsanity.com/paper/2607.10768/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use.
- Builds on 3 of its 39 references that are in the map.
- First posted on arXiv 12 Jul 2026.

## Placed in

- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)

## Builds on (most cited first, 3 of 3)

- [Synthesizing mixed-integer linear programming models from natural language descriptions](https://mlsanity.com/paper/2311.15271/) (2023)
- [OR-LLM-Agent: Automating Modeling and Solving of Operations Research Optimization Problems with Reasoning LLM](https://mlsanity.com/paper/2503.10009/) (2025)
- [OPT-Engine: Benchmarking the Limits of LLMs in Optimization Modeling via Complexity Scaling](https://mlsanity.com/paper/2601.19924/) (2026)

## Built on by (most cited first, 2 of 2)

- [LLMs as Adaptive Meta-Solvers: Strategy-Diverse RL for Industrial-Scale Optimization](https://mlsanity.com/paper/2609.34427/) (2026)
- [FDE-Bench: Evaluating End-to-End Delivery from Underspecified Real-World Business Requests](https://mlsanity.com/paper/2610.14539/) (2026)

## Abstract (arXiv)

LLM-based agents are increasingly deployed to solve optimization problems, yet existing benchmarks evaluate them on pre-structured mathematical formulations that bypass the most critical challenge: translating complex business requirements into correct models and solve efficiently. We introduce Opti-Agent-Bench, an end-to-end benchmark that evaluates Large Language Models (LLMs) across the complete optimization R&D pipeline, from understanding business-language descriptions through mathematical modeling, algorithm selection, and code implementation, to solution report generation. Our design rests on three pillars: (1) businesssemantic authenticity with anti-template traps that defeat pattern matching; (2) modular evaluation with cross-module consistency checking across Problem Understanding, Formal Modeling, Implementation, and Reporting; and (3) the ORAC bi-level validity framework that simultaneously ensures task quality and scoring integrity. Across several industrialscale tasks spanning integer programming, robust optimization, stochastic programming, and non-convex optimization, we expose critical failure modes of current models, including constraint omission, model-code inconsistency, and report-implementation divergence, that remain invisible under conventional single-metric evaluation.