# ConstraintLLM: A Neuro-Symbolic Framework for Industrial-Level Constraint Programming

arXiv: https://arxiv.org/abs/2510.05774 · HTML: https://mlsanity.com/paper/2510.05774/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 37 references that are in the map.
- First posted on arXiv 7 Oct 2025.

## Builds on (most cited first, 9 of 9)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models](https://mlsanity.com/paper/2403.13372/) (2024)
- [EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks](https://mlsanity.com/paper/1901.11196/) (2019)
- [Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoning](https://mlsanity.com/paper/2305.12295/) (2023)
- [LLMOPT: Learning to Define and Solve General Optimization Problems from Scratch](https://mlsanity.com/paper/2410.13213/) (2024)
- [Large Language Models Can Solve Real-World Planning Rigorously with Formal Verification Tools](https://mlsanity.com/paper/2404.11891/) (2024)
- [PyCSP3: Modeling Combinatorial Constrained Problems in Python](https://mlsanity.com/paper/2009.00326/) (2020)
- [Scalable Knowledge Refactoring using Constrained Optimisation](https://mlsanity.com/paper/2408.11530/) (2024)

## Built on by (most cited first, 10 of 10)

- [LLM-Guided Quantified SMT Solving over Uninterpreted Functions](https://mlsanity.com/paper/2601.04675/) (2026)
- [OptSkills: Learning Generalizable Optimization Skills from Problem Archetypes via Cluster-Based Distillation](https://mlsanity.com/paper/2605.29829/) (2026)
- [An Agentic Framework for Neuro-Symbolic Programming](https://mlsanity.com/paper/2601.00743/) (2026)
- [MaxSAT-Based Feedback for Guiding Vision-Language Models in Sudoku](https://mlsanity.com/paper/2607.12711/) (2026)
- [SCHEDBench: A Benchmark for Evaluating LLM Constraint Faithfulness in Natural-Language Combinatorial Scheduling](https://mlsanity.com/paper/2608.00991/) (2026)
- [Reliable Reasoning with Large Language Models via Preference-Based Maximum Satisfiability](https://mlsanity.com/paper/2605.29687/) (2026)
- [LLM-Guided Evolutionary Search for Constraint Model Reformulation to Improve Solver Efficiency](https://mlsanity.com/paper/2607.28268/) (2026)
- [Improving Natural-Language Combinatorial-Optimization Accuracy in Resource-Constrained Language Models via Formal Abstractions](https://mlsanity.com/paper/2608.18409/) (2026)
- [Language-Augmented Semantic Priors for B-Spline Surface Fitting](https://mlsanity.com/paper/2609.11708/) (2026)
- [Neuro-symbolic AI for Industrial Configuration](https://mlsanity.com/paper/2609.29947/) (2026)

## Abstract (arXiv)

Constraint programming (CP) is a crucial technology for solving real-world constraint optimization problems (COPs), with the advantages of rich modeling semantics and high solving efficiency. Using large language models (LLMs) to generate formal modeling automatically for COPs is becoming a promising approach, which aims to build trustworthy neuro-symbolic AI with the help of symbolic solvers. However, CP has received less attention compared to works based on operations research (OR) models. We introduce ConstraintLLM, the first LLM specifically designed for CP modeling, which is trained on an open-source LLM with multi-instruction supervised fine-tuning. We propose the Constraint-Aware Retrieval Module (CARM) to increase the in-context learning capabilities, which is integrated in a Tree-of-Thoughts (ToT) framework with guided self-correction mechanism. Moreover, we construct and release IndusCP, the first industrial-level benchmark for CP modeling, which contains 140 challenging tasks from various domains. Our experiments demonstrate that ConstraintLLM achieves state-of-the-art solving accuracy across multiple benchmarks and outperforms the baselines by 2x on the new IndusCP benchmark. Code and data are available at: https://github.com/william4s/ConstraintLLM.