# Learning programs by learning from failures

arXiv: https://arxiv.org/abs/2005.02259 · HTML: https://mlsanity.com/paper/2005.02259/

## Key facts

- Cited by 73 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 May 2020.

## Built on by (most cited first, 30 of 73)

- [Inductive logic programming at 30](https://mlsanity.com/paper/2102.10556/) (2021)
- [Neuro-Symbolic Hierarchical Rule Induction](https://mlsanity.com/paper/2112.13418/) (2021)
- [CrossBeam: Learning to Search in Bottom-Up Program Synthesis](https://mlsanity.com/paper/2203.10452/) (2022)
- [Imperative Learning: A Self-supervised Neuro-Symbolic Learning Framework for Robot Autonomy](https://mlsanity.com/paper/2406.16087/) (2024)
- [Learning MDL logic programs from noisy data](https://mlsanity.com/paper/2308.09393/) (2023)
- [Learning Higher-Order Programs without Meta-Interpretive Learning](https://mlsanity.com/paper/2112.14603/) (2021)
- [Predicate Invention by Learning From Failures](https://mlsanity.com/paper/2104.14426/) (2021)
- [Interpretable Model-based Hierarchical Reinforcement Learning using Inductive Logic Programming](https://mlsanity.com/paper/2106.11417/) (2021)
- [V-LoL: A Diagnostic Dataset for Visual Logical Learning](https://mlsanity.com/paper/2306.07743/) (2023)
- [LogiCity: Advancing Neuro-Symbolic AI with Abstract Urban Simulation](https://mlsanity.com/paper/2411.00773/) (2024)
- [Towards General Natural Language Understanding with Probabilistic Worldbuilding](https://mlsanity.com/paper/2105.02486/) (2021)
- [A Critical Review of Inductive Logic Programming Techniques for Explainable AI](https://mlsanity.com/paper/2112.15319/) (2021)
- [Learning logic programs by combining programs](https://mlsanity.com/paper/2206.01614/) (2022)
- [Anti-unification and Generalization: A Survey](https://mlsanity.com/paper/2302.00277/) (2023)
- [FormaT5: Abstention and Examples for Conditional Table Formatting with Natural Language](https://mlsanity.com/paper/2310.17306/) (2023)
- [Relational decomposition for program synthesis](https://mlsanity.com/paper/2408.12212/) (2024)
- [Learning Logic Programs From Noisy Failures](https://mlsanity.com/paper/2201.03702/) (2021)
- [Generalisation Through Negation and Predicate Invention](https://mlsanity.com/paper/2301.07629/) (2023)
- [Towards Probabilistic Inductive Logic Programming with Neurosymbolic Inference and Relaxation](https://mlsanity.com/paper/2408.11367/) (2024)
- [Rule Induction in Knowledge Graphs Using Linear Programming](https://mlsanity.com/paper/2110.08245/) (2021)
- [Learning logic programs by discovering where not to search](https://mlsanity.com/paper/2202.09806/) (2022)
- [CORNET: Learning Table Formatting Rules By Example](https://mlsanity.com/paper/2208.06032/) (2022)
- [Learning big logical rules by joining small rules](https://mlsanity.com/paper/2401.16215/) (2024)
- [Online inductive learning from answer sets for efficient reinforcement learning exploration](https://mlsanity.com/paper/2501.07445/) (2025)
- [Abductive Logical Rule Induction by Bridging Inductive Logic Programming and Multimodal Large Language Models](https://mlsanity.com/paper/2509.21874/) (2025)
- [Meta-Interpretive Learning as Metarule Specialisation](https://mlsanity.com/paper/2106.07464/) (2021)
- [Efficient lifting of symmetry breaking constraints for complex combinatorial problems](https://mlsanity.com/paper/2205.07129/) (2022)
- [Learning programs with magic values](https://mlsanity.com/paper/2208.03238/) (2022)
- [Relational program synthesis with numerical reasoning](https://mlsanity.com/paper/2210.00764/) (2022)
- [Inductive Learning of Robot Task Knowledge from Raw Data and Online Expert Feedback](https://mlsanity.com/paper/2501.07507/) (2025)

## Abstract (arXiv)

We describe an inductive logic programming (ILP) approach called learning from failures. In this approach, an ILP system (the learner) decomposes the learning problem into three separate stages: generate, test, and constrain. In the generate stage, the learner generates a hypothesis (a logic program) that satisfies a set of hypothesis constraints (constraints on the syntactic form of hypotheses). In the test stage, the learner tests the hypothesis against training examples. A hypothesis fails when it does not entail all the positive examples or entails a negative example. If a hypothesis fails, then, in the constrain stage, the learner learns constraints from the failed hypothesis to prune the hypothesis space, i.e. to constrain subsequent hypothesis generation. For instance, if a hypothesis is too general (entails a negative example), the constraints prune generalisations of the hypothesis. If a hypothesis is too specific (does not entail all the positive examples), the constraints prune specialisations of the hypothesis. This loop repeats until either (i) the learner finds a hypothesis that entails all the positive and none of the negative examples, or (ii) there are no more hypotheses to test. We introduce Popper, an ILP system that implements this approach by combining answer set programming and Prolog. Popper supports infinite problem domains, reasoning about lists and numbers, learning textually minimal programs, and learning recursive programs. Our experimental results on three domains (toy game problems, robot strategies, and list transformations) show that (i) constraints drastically improve learning performance, and (ii) Popper can outperform existing ILP systems, both in terms of predictive accuracies and learning times.