Ask Before You Optimize: Dynamic Pre-Formulation Clarification for Interactive Optimization
Not on the map. Why?
Abstract (arXiv)
Large language models (LLMs) are increasingly used to formulate optimization models from natural-language problem descriptions, yet realistic operations research (OR) requests are often incomplete: missing objectives, constraints, or business rules can change the resulting mathematical program. Existing evaluations largely assume a complete specification and therefore overlook whether an agent knows when clarification is needed before modeling. We introduce OR-Clarify, a benchmark for pre-formulation clarification. Each task presents a partial public problem description, withholds structured hidden slots, and evaluates agents through bounded interaction with a simulated user. The benchmark supports both openended and choice-based clarification, and measures slot recovery, stopping behavior, silent assumptions, and interaction cost. We further propose Interactive Optimization (InterOPT), a two-stage framework that identifies unresolved formulation-critical gaps and uses them to guide whether to ask the next question or to stop. In our choice-based experiments, InterOPT substantially outperforms all baselines in exact slot recovery; in the open-ended setting, it remains competitive with strong prior methods. Together, OR-Clarify and InterOPT reframe OR assistance as a selective completeness decision: clarify when needed, stop when ready, and quantify what remains missing.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 26 of its 29 references that are in the map. First posted on arXiv 4 Sep 2026.
Builds on
26 of 29 references.
References · 26
- 2022Language Models (Mostly) Know What They Knowcited by 1,856
- 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,402
- 2020AmbigQA: Answering Ambiguous Open-domain Questionscited by 390
- 2025AbstentionBench: Reasoning LLMs Fail on Unanswerable Questionscited by 145
- 2019Asking Clarifying Questions in Open-Domain Information-Seeking Conversationscited by 139
- 2023NL4Opt Competition: Formulating Optimization Problems Based on Their Natural Language Descriptionscited by 135
- 2024ORLM: A Customizable Framework in Training Large Models for Automated Optimization Modelingcited by 124
- 2024OptiMUS: Scalable Optimization Modeling with (MI)LP Solvers and Large Language Modelscited by 109
- 2024STaR-GATE: Teaching Language Models to Ask Clarifying Questionscited by 104
- 2024LLMOPT: Learning to Define and Solve General Optimization Problems from Scratchcited by 102
- 2023Clarify When Necessary: Resolving Ambiguity Through Interaction with LMscited by 98
- 2023Eliciting Human Preferences with Language Modelscited by 92
14 more
- 2024Modeling Future Conversation Turns to Teach LLMs to Ask Clarifying Questionscited by 76
- 2018Interpretation of Natural Language Rules in Conversational Machine Readingcited by 75
- 2022CLAM: Selective Clarification for Ambiguous Questions with Generative Language Modelscited by 69
- 2020ConvAI3: Generating Clarifying Questions for Open-Domain Dialogue Systems (ClariQ)cited by 58
- 2025Structured Uncertainty guided Clarification for LLM Agentscited by 41
- 2023Asking Clarification Questions to Handle Ambiguity in Open-Domain QAcited by 26
- 2025StepORLM: A Self-Evolving Framework With Generative Process Supervision For Operations Research Language Modelscited by 26
- 2026When and What to Ask: AskBench and Rubric-Guided RLVR for LLM Clarificationcited by 16
- 2026Large-Scale Optimization Model Auto-Formulation: Harnessing LLM Flexibility via Structured Workflowcited by 13
- 2025TO-GATE: Clarifying Questions and Summarizing Responses with Trajectory Optimization for Eliciting Human Preferencecited by 5
- 2026Let's Have a Conversation: Designing and Evaluating LLM Agents for Interactive Optimizationcited by 3
- 2026CLARITY: A Framework and Benchmark for Conversational Language Ambiguity and Unanswerability in Interactive NL2SQL Systemscited by 2
- 2026ORPilot: A Production-Oriented Agentic LLM-for-OR Tool for Optimization Modelingcited by 2
- 2026OR-Space: A Full-Lifecycle Workspace Benchmark for Industrial Optimization Agentscited by 2
Built on by
Not cited yet.