# Differentiable Rule Induction with Learned Relational Features

arXiv: https://arxiv.org/abs/2201.06515 · HTML: https://mlsanity.com/paper/2201.06515/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 30 references that are in the map.
- First posted on arXiv 17 Jan 2022.

## Builds on (most cited first, 17 of 17)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation](https://mlsanity.com/paper/1308.3432/) (2013)
- [Neural Probabilistic Logic Programming in DeepProbLog](https://mlsanity.com/paper/1907.08194/) (2019)
- [Understanding the Role of Individual Units in a Deep Neural Network](https://mlsanity.com/paper/2009.05041/) (2020)
- [Learning Explanatory Rules from Noisy Data](https://mlsanity.com/paper/1711.04574/) (2017)
- [Boolean Decision Rules via Column Generation](https://mlsanity.com/paper/1805.09901/) (2018)
- [Estimating or Propagating Gradients Through Stochastic Neurons](https://mlsanity.com/paper/1305.2982/) (2013)
- [Learning Certifiably Optimal Rule Lists for Categorical Data](https://mlsanity.com/paper/1704.01701/) (2017)
- [Scalable Rule-Based Representation Learning for Interpretable Classification](https://mlsanity.com/paper/2109.15103/) (2021)
- [Learning Accurate and Interpretable Decision Rule Sets from Neural Networks](https://mlsanity.com/paper/2103.02826/) (2021)
- [A Scalable MIP-based Method for Learning Optimal Multivariate Decision Trees](https://mlsanity.com/paper/2011.03375/) (2020)
- [Relational Neural Machines](https://mlsanity.com/paper/2002.02193/) (2020)
- [Symbolic Logic meets Machine Learning: A Brief Survey in Infinite Domains](https://mlsanity.com/paper/2006.08480/) (2020)
- [Lifted Relational Neural Networks](https://mlsanity.com/paper/1508.05128/) (2015)
- [Learning Structured Declarative Rule Sets -- A Challenge for Deep Discrete Learning](https://mlsanity.com/paper/2012.04377/) (2020)
- [Demystifying Relational Latent Representations](https://mlsanity.com/paper/1705.05785/) (2017)
- [An Investigation into Mini-Batch Rule Learning](https://mlsanity.com/paper/2106.10202/) (2021)

## Built on by (most cited first, 8 of 8)

- [DeepDFA: Automata Learning through Neural Probabilistic Relaxations](https://mlsanity.com/paper/2408.08622/) (2024)
- [Neural Reward Machines](https://mlsanity.com/paper/2408.08677/) (2024)
- [Tackling Interpretability in Audio Classification Networks with Non-negative Matrix Factorization](https://mlsanity.com/paper/2305.07132/) (2023)
- [Finding Interpretable Class-Specific Patterns through Efficient Neural Search](https://mlsanity.com/paper/2312.04311/) (2023)
- [Neural-based classification rule learning for sequential data](https://mlsanity.com/paper/2302.11286/) (2023)
- [RIFF: Inducing Rules for Fraud Detection from Decision Trees](https://mlsanity.com/paper/2408.12989/) (2024)
- [Differentiable Inductive Logic Programming for Fraud Detection](https://mlsanity.com/paper/2410.21928/) (2024)
- [Logic of Hypotheses: from Zero to Full Knowledge in Neurosymbolic Integration](https://mlsanity.com/paper/2509.21663/) (2025)

## Abstract (arXiv)

Rule-based decision models are attractive due to their interpretability. However, existing rule induction methods often result in long and consequently less interpretable rule models. This problem can often be attributed to the lack of appropriately expressive vocabulary, i.e., relevant predicates used as literals in the decision model. Most existing rule induction algorithms presume pre-defined literals, naturally decoupling the definition of the literals from the rule learning phase. In contrast, we propose the Relational Rule Network (R2N), a neural architecture that learns literals that represent a linear relationship among numerical input features along with the rules that use them. This approach opens the door to increasing the expressiveness of induced decision models by coupling literal learning directly with rule learning in an end-to-end differentiable fashion. On benchmark tasks, we show that these learned literals are simple enough to retain interpretability, yet improve prediction accuracy and provide sets of rules that are more concise compared to state-of-the-art rule induction algorithms.