# Holy Grail 2.0: From Natural Language to Constraint Models

arXiv: https://arxiv.org/abs/2308.01589 · HTML: https://mlsanity.com/paper/2308.01589/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 26 references that are in the map.
- First posted on arXiv 3 Aug 2023.

## Builds on (most cited first, 17 of 17)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021)
- [Tree of Thoughts: Deliberate Problem Solving with Large Language Models](https://mlsanity.com/paper/2305.10601/) (2023)
- [Scaling Language Models: Methods, Analysis & Insights from Training Gopher](https://mlsanity.com/paper/2112.11446/) (2021)
- [Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigm](https://mlsanity.com/paper/2102.07350/) (2021)
- [Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Models](https://mlsanity.com/paper/2305.04091/) (2023)
- [Fine-Tuning Pretrained Language Models: Weight Initializations, Data Orders, and Early Stopping](https://mlsanity.com/paper/2002.06305/) (2020)
- [A Systematic Evaluation of Large Language Models of Code](https://mlsanity.com/paper/2202.13169/) (2022)
- [Large Language Model Guided Tree-of-Thought](https://mlsanity.com/paper/2305.08291/) (2023)
- [NL4Opt Competition: Formulating Optimization Problems Based on Their Natural Language Descriptions](https://mlsanity.com/paper/2303.08233/) (2023)
- [An Analysis of the Automatic Bug Fixing Performance of ChatGPT](https://mlsanity.com/paper/2301.08653/) (2023)
- [Augmenting Operations Research with Auto-Formulation of Optimization Models from Problem Descriptions](https://mlsanity.com/paper/2209.15565/) (2022)
- [Highlighting Named Entities in Input for Auto-Formulation of Optimization Problems](https://mlsanity.com/paper/2212.13201/) (2022)
- [A Novel Approach for Auto-Formulation of Optimization Problems](https://mlsanity.com/paper/2302.04643/) (2023)
- [Towards an Automatic Optimisation Model Generator Assisted with Generative Pre-trained Transformer](https://mlsanity.com/paper/2305.05811/) (2023)

## Built on by (most cited first, 26 of 26)

- [Solver-Informed RL: Grounding Large Language Models for Authentic Optimization Modeling](https://mlsanity.com/paper/2505.11792/) (2025)
- [LM4OPT: Unveiling the Potential of Large Language Models in Formulating Mathematical Optimization Problems](https://mlsanity.com/paper/2403.01342/) (2024)
- [From Large Language Models and Optimization to Decision Optimization CoPilot: A Research Manifesto](https://mlsanity.com/paper/2402.16269/) (2024)
- [A Systematic Survey on Large Language Models for Evolutionary Optimization: From Modeling to Solving](https://mlsanity.com/paper/2509.08269/) (2025)
- [Combining Constraint Programming Reasoning with Large Language Model Predictions](https://mlsanity.com/paper/2407.13490/) (2024)
- [LLMs for Cold-Start Cutting Plane Separator Configuration](https://mlsanity.com/paper/2412.12038/) (2024)
- [SAC-Opt: Semantic Anchors for Iterative Correction in Optimization Modeling](https://mlsanity.com/paper/2510.05115/) (2025)
- [Text2Zinc: A Cross-Domain Dataset for Modeling Optimization and Satisfaction Problems in MiniZinc](https://mlsanity.com/paper/2503.10642/) (2025)
- [CP-Agent: Agentic Constraint Programming](https://mlsanity.com/paper/2508.07468/) (2025)
- [Abstract Operations Research Modeling Using Natural Language Inputs](https://mlsanity.com/paper/2408.07272/) (2024)
- [DualSchool: How Reliable are LLMs for Optimization Education?](https://mlsanity.com/paper/2505.21775/) (2025)
- [DCP-Bench-Open: Evaluating LLMs for Constraint Modelling of Discrete Combinatorial Problems](https://mlsanity.com/paper/2506.06052/) (2025)
- [Distributed Multi-Agent Coordination Using Multi-Modal Foundation Models](https://mlsanity.com/paper/2501.14189/) (2025)
- [CHORUS: Zero-shot Hierarchical Retrieval and Orchestration for Generating Linear Programming Code](https://mlsanity.com/paper/2505.01485/) (2025)
- [LogiPlan: A Structured Benchmark for Logical Planning and Relational Reasoning in LLMs](https://mlsanity.com/paper/2506.10527/) (2025)
- [Gala: Global LLM Agents for Text-to-Model Translation](https://mlsanity.com/paper/2509.08970/) (2025)
- [MM-OptBench: A Solver-Grounded Benchmark for Multimodal Optimization Modeling](https://mlsanity.com/paper/2605.12154/) (2026)
- [TriVAL: A Tri-Validation Framework for Faithful Automatic Optimization Modeling](https://mlsanity.com/paper/2605.23966/) (2026)
- [Behavior and Representation in Open-Weight Large Language Models for Combinatorial Optimization: From Feature Extraction to Algorithm Selection](https://mlsanity.com/paper/2512.13374/) (2025)
- [Interactionless Inverse Reinforcement Learning: A Data-Centric Framework for Durable Alignment](https://mlsanity.com/paper/2602.14844/) (2026)
- [Automatic Feature Learning for Essence: a Case Study on Car Sequencing](https://mlsanity.com/paper/2409.15158/) (2024)
- [Optimization Problem Solving Can Transition to Evolutionary Agentic Workflows](https://mlsanity.com/paper/2505.04354/) (2025)
- [CP-Model-Zoo: A Natural Language Query System for Constraint Programming Models](https://mlsanity.com/paper/2509.07867/) (2025)
- [LM4Opt-RA: A Multi-Candidate LLM Framework with Structured Ranking for Automating Network Resource Allocation](https://mlsanity.com/paper/2512.00039/) (2025)
- [Learn2Zinc: Fine-tuning Small Language Models for Text-to-Model Translation in MiniZinc](https://mlsanity.com/paper/2607.20456/) (2026)
- [Auto-Formalizing Neuro-Symbolic Predictors](https://mlsanity.com/paper/2610.01519/) (2026)

## Abstract (arXiv)

Twenty-seven years ago, E. Freuder highlighted that "Constraint programming represents one of the closest approaches computer science has yet made to the Holy Grail of programming: the user states the problem, the computer solves it". Nowadays, CP users have great modeling tools available (like Minizinc and CPMpy), allowing them to formulate the problem and then let a solver do the rest of the job, getting closer to the stated goal. However, this still requires the CP user to know the formalism and respect it. Another significant challenge lies in the expertise required to effectively model combinatorial problems. All this limits the wider adoption of CP. In this position paper, we investigate a possible approach to leverage pre-trained Large Language Models to extract models from textual problem descriptions. More specifically, we take inspiration from the Natural Language Processing for Optimization (NL4OPT) challenge and present early results with a decomposition-based prompting approach to GPT Models.