# Neural Rule Ensembles: Encoding Sparse Feature Interactions into Neural Networks

arXiv: https://arxiv.org/abs/2002.04319 · HTML: https://mlsanity.com/paper/2002.04319/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Feb 2020.

## Built on by (most cited first, 3 of 3)

- [Online Feature Screening for Data Streams with Concept Drift](https://mlsanity.com/paper/2104.02883/) (2021)
- [Exploring Vision Neural Network Pruning via Screening Methodology](https://mlsanity.com/paper/2502.07189/) (2025)
- [Sparse Oblique Rule Boosting for Simpler Additive Rule Ensembles](https://mlsanity.com/paper/2609.06426/) (2026)

## Abstract (arXiv)

Artificial Neural Networks form the basis of very powerful learning methods. It has been observed that a naive application of fully connected neural networks to data with many irrelevant variables often leads to overfitting. In an attempt to circumvent this issue, a prior knowledge pertaining to what features are relevant and their possible feature interactions can be encoded into these networks. In this work, we use decision trees to capture such relevant features and their interactions and define a mapping to encode extracted relationships into a neural network. This addresses the initialization related concern of fully connected neural networks. At the same time through feature selection it enables learning of compact representations compared to state of the art tree-based approaches. Empirical evaluations and simulation studies show the superiority of such an approach over fully connected neural networks and tree-based approaches