# Learning Gradient Boosted Multi-label Classification Rules

arXiv: https://arxiv.org/abs/2006.13346 · HTML: https://mlsanity.com/paper/2006.13346/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Jun 2020.

## Built on by (most cited first, 11 of 11)

- [Efficient learning of large sets of locally optimal classification rules](https://mlsanity.com/paper/2301.09936/) (2023)
- [An Empirical Investigation into Deep and Shallow Rule Learning](https://mlsanity.com/paper/2106.10254/) (2021)
- [Tree-Based Dynamic Classifier Chains](https://mlsanity.com/paper/2112.06672/) (2021)
- [pRSL: Interpretable Multi-label Stacking by Learning Probabilistic Rules](https://mlsanity.com/paper/2105.13850/) (2021)
- [An Investigation into Mini-Batch Rule Learning](https://mlsanity.com/paper/2106.10202/) (2021)
- [Gradient-based Label Binning in Multi-label Classification](https://mlsanity.com/paper/2106.11690/) (2021)
- [Concise and interpretable multi-label rule sets](https://mlsanity.com/paper/2210.01533/) (2022)
- [Correlation-based Discovery of Disease Patterns for Syndromic Surveillance](https://mlsanity.com/paper/2110.09208/) (2021)
- [Minimal Learning Machine for Multi-Label Learning](https://mlsanity.com/paper/2305.05518/) (2023)
- [RuleKit 2: Faster and simpler rule learning](https://mlsanity.com/paper/2504.20650/) (2025)
- [On Trustworthy Rule-Based Models and Explanations](https://mlsanity.com/paper/2507.07576/) (2025)

## Abstract (arXiv)

In multi-label classification, where the evaluation of predictions is less straightforward than in single-label classification, various meaningful, though different, loss functions have been proposed. Ideally, the learning algorithm should be customizable towards a specific choice of the performance measure. Modern implementations of boosting, most prominently gradient boosted decision trees, appear to be appealing from this point of view. However, they are mostly limited to single-label classification, and hence not amenable to multi-label losses unless these are label-wise decomposable. In this work, we develop a generalization of the gradient boosting framework to multi-output problems and propose an algorithm for learning multi-label classification rules that is able to minimize decomposable as well as non-decomposable loss functions. Using the well-known Hamming loss and subset 0/1 loss as representatives, we analyze the abilities and limitations of our approach on synthetic data and evaluate its predictive performance on multi-label benchmarks.