# Teaching the Old Dog New Tricks: Supervised Learning with Constraints

arXiv: https://arxiv.org/abs/2002.10766 · HTML: https://mlsanity.com/paper/2002.10766/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Feb 2020.

## Built on by (most cited first, 16 of 16)

- [End-to-End Constrained Optimization Learning: A Survey](https://mlsanity.com/paper/2103.16378/) (2021)
- [Modular Design Patterns for Hybrid Learning and Reasoning Systems: a taxonomy, patterns and use cases](https://mlsanity.com/paper/2102.11965/) (2021)
- [Strong Optimal Classification Trees](https://mlsanity.com/paper/2103.15965/) (2021)
- [Learning Hard Optimization Problems: A Data Generation Perspective](https://mlsanity.com/paper/2106.02601/) (2021)
- [A New Computationally Simple Approach for Implementing Neural Networks with Output Hard Constraints](https://mlsanity.com/paper/2307.10459/) (2023)
- [Zero-Regret Performative Prediction Under Inequality Constraints](https://mlsanity.com/paper/2309.12618/) (2023)
- [Generalized Disparate Impact for Configurable Fairness Solutions in ML](https://mlsanity.com/paper/2305.18504/) (2023)
- [PROPEL: Supervised and Reinforcement Learning for Large-Scale Supply Chain Planning](https://mlsanity.com/paper/2504.07383/) (2025)
- [Knowledge Enhanced Neural Networks for relational domains](https://mlsanity.com/paper/2205.15762/) (2022)
- [Constrained Imitation Learning for a Flapping Wing Unmanned Aerial Vehicle](https://mlsanity.com/paper/2206.03987/) (2022)
- [Constrained Machine Learning: The Bagel Framework](https://mlsanity.com/paper/2112.01088/) (2021)
- [Integrating Symbolic Reasoning into Neural Generative Models for Design Generation](https://mlsanity.com/paper/2310.09383/) (2023)
- [The intersection of machine learning with forecasting and optimisation: theory and applications](https://mlsanity.com/paper/2211.13583/) (2022)
- [Approximating Solutions to the Knapsack Problem using the Lagrangian Dual Framework](https://mlsanity.com/paper/2312.03413/) (2023)
- [Imitation Learning for Combinatorial Optimisation under Uncertainty](https://mlsanity.com/paper/2601.05383/) (2026)
- [Fusing Backdoors, Machine Learning, and Optimization for Large-Scale Parametric Mixed-Integer Programs](https://mlsanity.com/paper/2606.21440/) (2026)

## Abstract (arXiv)

Adding constraint support in Machine Learning has the potential to address outstanding issues in data-driven AI systems, such as safety and fairness. Existing approaches typically apply constrained optimization techniques to ML training, enforce constraint satisfaction by adjusting the model design, or use constraints to correct the output. Here, we investigate a different, complementary, strategy based on "teaching" constraint satisfaction to a supervised ML method via the direct use of a state-of-the-art constraint solver: this enables taking advantage of decades of research on constrained optimization with limited effort. In practice, we use a decomposition scheme alternating master steps (in charge of enforcing the constraints) and learner steps (where any supervised ML model and training algorithm can be employed). The process leads to approximate constraint satisfaction in general, and convergence properties are difficult to establish; despite this fact, we found empirically that even a na"ive setup of our approach performs well on ML tasks with fairness constraints, and on classical datasets with synthetic constraints.