# Binarsity: a penalization for one-hot encoded features in linear supervised learning

arXiv: https://arxiv.org/abs/1703.08619 · HTML: https://mlsanity.com/paper/1703.08619/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Mar 2017.

## Built on by (most cited first, 4 of 4)

- [Does Configuration Encoding Matter in Learning Software Performance? An Empirical Study on Encoding Schemes](https://mlsanity.com/paper/2203.15988/) (2022)
- [Deep Configuration Performance Learning: A Systematic Survey and Taxonomy](https://mlsanity.com/paper/2403.03322/) (2024)
- [Modeling High-Dimensional Data with Unknown Cut Points: A Fusion Penalized Logistic Threshold Regression](https://mlsanity.com/paper/2202.08441/) (2022)
- [Estimation of an Order Book Dependent Hawkes Process for Large Datasets](https://mlsanity.com/paper/2307.09077/) (2023)

## Abstract (arXiv)

This paper deals with the problem of large-scale linear supervised learning in settings where a large number of continuous features are available. We propose to combine the well-known trick of one-hot encoding of continuous features with a new penalization called binarsity. In each group of binary features coming from the one-hot encoding of a single raw continuous feature, this penalization uses total-variation regularization together with an extra linear constraint. This induces two interesting properties on the model weights of the one-hot encoded features: they are piecewise constant, and are eventually block sparse. Non-asymptotic oracle inequalities for generalized linear models are proposed. Moreover, under a sparse additive model assumption, we prove that our procedure matches the state-of-the-art in this setting. Numerical experiments illustrate the good performances of our approach on several datasets. It is also noteworthy that our method has a numerical complexity comparable to standard ℓ1 penalization.