# A Solver-Free Framework for Scalable Learning in Neural ILP Architectures arXiv: https://arxiv.org/abs/2210.09082 ยท HTML: https://mlsanity.com/paper/2210.09082/ ## Key facts - Cited by 5 papers in the map (data through 14 Oct 2026). - Builds on 32 of its 66 references that are in the map. - First posted on arXiv 17 Oct 2022. ## Builds on (most cited first, 30 of 32) - [OptNet: Differentiable Optimization as a Layer in Neural Networks](https://mlsanity.com/paper/1703.00443/) (2017) - [Differentiable Convex Optimization Layers](https://mlsanity.com/paper/1910.12430/) (2019) - [Supervised Learning of Universal Sentence Representations from Natural Language Inference Data](https://mlsanity.com/paper/1705.02364/) (2017) - [Melding the Data-Decisions Pipeline: Decision-Focused Learning for Combinatorial Optimization](https://mlsanity.com/paper/1809.05504/) (2018) - [Differentiation of Blackbox Combinatorial Solvers](https://mlsanity.com/paper/1912.02175/) (2019) - [Learning a SAT Solver from Single-Bit Supervision](https://mlsanity.com/paper/1802.03685/) (2018) - [SATNet: Bridging deep learning and logical reasoning using a differentiable satisfiability solver](https://mlsanity.com/paper/1905.12149/) (2019) - [SPair-71k: A Large-scale Benchmark for Semantic Correspondence](https://mlsanity.com/paper/1908.10543/) (2019) - [Neural Logic Machines](https://mlsanity.com/paper/1904.11694/) (2019) - [Implicit MLE: Backpropagating Through Discrete Exponential Family Distributions](https://mlsanity.com/paper/2106.01798/) (2021) - [MIPaaL: Mixed Integer Program as a Layer](https://mlsanity.com/paper/1907.05912/) (2019) - [Interior Point Solving for LP-based prediction+optimisation](https://mlsanity.com/paper/2010.13943/) (2020) - [Deep Graph Matching via Blackbox Differentiation of Combinatorial Solvers](https://mlsanity.com/paper/2003.11657/) (2020) - [Recurrent Relational Networks](https://mlsanity.com/paper/1711.08028/) (2017) - [Optimizing Rank-based Metrics with Blackbox Differentiation](https://mlsanity.com/paper/1912.03500/) (2019) - [Joint Event and Temporal Relation Extraction with Shared Representations and Structured Prediction](https://mlsanity.com/paper/1909.05360/) (2019) - [Augmenting Neural Networks with First-order Logic](https://mlsanity.com/paper/1906.06298/) (2019) - [CombOptNet: Fit the Right NP-Hard Problem by Learning Integer Programming Constraints](https://mlsanity.com/paper/2105.02343/) (2021) - [Multi-Loss Weighting with Coefficient of Variations](https://mlsanity.com/paper/2009.01717/) (2020) - [An Improved Neural Baseline for Temporal Relation Extraction](https://mlsanity.com/paper/1909.00429/) (2019) - [GREED: A Neural Framework for Learning Graph Distance Functions](https://mlsanity.com/paper/2112.13143/) (2021) - [Learning Linear Programs from Optimal Decisions](https://mlsanity.com/paper/2006.08923/) (2020) - [PDP: A General Neural Framework for Learning Constraint Satisfaction Solvers](https://mlsanity.com/paper/1903.01969/) (2019) - [Neural Learning of One-of-Many Solutions for Combinatorial Problems in Structured Output Spaces](https://mlsanity.com/paper/2008.11990/) (2020) - [Size Independent Neural Transfer for RDDL Planning](https://mlsanity.com/paper/1902.03081/) (2019) - [Transfer of Deep Reactive Policies for MDP Planning](https://mlsanity.com/paper/1810.11488/) (2018) - [Expressiveness of Rectifier Networks](https://mlsanity.com/paper/1511.05678/) (2015) - [Deep Inverse Optimization](https://mlsanity.com/paper/1812.00804/) (2018) - [End-to-End Neuro-Symbolic Architecture for Image-to-Image Reasoning Tasks](https://mlsanity.com/paper/2106.03121/) (2021) - [Learning Constraints for Structured Prediction Using Rectifier Networks](https://mlsanity.com/paper/2006.01209/) (2020) ## Built on by (most cited first, 5 of 5) - [Decision-Focused Learning: Foundations, State of the Art, Benchmark and Future Opportunities](https://mlsanity.com/paper/2307.13565/) (2023) - [Feasibility-Aware Decision-Focused Learning for Predicting Parameters in the Constraints](https://mlsanity.com/paper/2510.04951/) (2025) - [Two-Stage Predict+Optimize for Mixed Integer Linear Programs with Unknown Parameters in Constraints](https://mlsanity.com/paper/2311.08022/) (2023) - [Entropy Can Flow, or It Can Guide. Be Entropy. LEDFlow: Introducing Entropy-guided Generation Order into Uniform Discrete Flow](https://mlsanity.com/paper/2609.25131/) (2026) - [PolyStepOR: Learning to Decide Without Optimal Decisions](https://mlsanity.com/paper/2609.32465/) (2026) ## Abstract (arXiv) There is a recent focus on designing architectures that have an Integer Linear Programming (ILP) layer within a neural model (referred to as Neural ILP in this paper). Neural ILP architectures are suitable for pure reasoning tasks that require data-driven constraint learning or for tasks requiring both perception (neural) and reasoning (ILP). A recent SOTA approach for end-to-end training of Neural ILP explicitly defines gradients through the ILP black box (Paulus et al. 2021) - this trains extremely slowly, owing to a call to the underlying ILP solver for every training data point in a minibatch. In response, we present an alternative training strategy that is solver-free, i.e., does not call the ILP solver at all at training time. Neural ILP has a set of trainable hyperplanes (for cost and constraints in ILP), together representing a polyhedron. Our key idea is that the training loss should impose that the final polyhedron separates the positives (all constraints satisfied) from the negatives (at least one violated constraint or a suboptimal cost value), via a soft-margin formulation. While positive example(s) are provided as part of the training data, we devise novel techniques for generating negative samples. Our solution is flexible enough to handle equality as well as inequality constraints. Experiments on several problems, both perceptual as well as symbolic, which require learning the constraints of an ILP, show that our approach has superior performance and scales much better compared to purely neural baselines and other state-of-the-art models that require solver-based training. In particular, we are able to obtain excellent performance in 9 x 9 symbolic and visual sudoku, to which the other Neural ILP solver is not able to scale.