# Quantile Encoder: Tackling High Cardinality Categorical Features in Regression Problems

arXiv: https://arxiv.org/abs/2105.13783 · HTML: https://mlsanity.com/paper/2105.13783/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 1 of its 27 references that are in the map.
- First posted on arXiv 27 May 2021.

## Builds on (most cited first, 1 of 1)

- [Encoding Categorical Variables with Conjugate Bayesian Models for WeWork Lead Scoring Engine](https://mlsanity.com/paper/1904.13001/) (2019)

## Built on by (most cited first, 5 of 5)

- [Fairness Implications of Encoding Protected Categorical Attributes](https://mlsanity.com/paper/2201.11358/) (2022)
- [Comparative Study on the Performance of Categorical Variable Encoders in Classification and Regression Tasks](https://mlsanity.com/paper/2401.09682/) (2024)
- [Tabular Learning: Encoding for Entity and Context Embeddings](https://mlsanity.com/paper/2403.19405/) (2024)
- [Introducing explainable supervised machine learning into interactive feedback loops for statistical production system](https://mlsanity.com/paper/2202.03212/) (2022)
- [End-to-End Optimized Pipeline for Prediction of Protein Folding Kinetics](https://mlsanity.com/paper/2309.09191/) (2023)

## Abstract (arXiv)

Regression problems have been widely studied in machinelearning literature resulting in a plethora of regression models and performance measures. However, there are few techniques specially dedicated to solve the problem of how to incorporate categorical features to regression problems. Usually, categorical feature encoders are general enough to cover both classification and regression problems. This lack of specificity results in underperforming regression models. In this paper,we provide an in-depth analysis of how to tackle high cardinality categor-ical features with the quantile. Our proposal outperforms state-of-the-encoders, including the traditional statistical mean target encoder, when considering the Mean Absolute Error, especially in the presence of long-tailed or skewed distributions. Besides, to deal with possible overfitting when there are categories with small support, our encoder benefits from additive smoothing. Finally, we describe how to expand the encoded values by creating a set of features with different quantiles. This expanded encoder provides a more informative output about the categorical feature in question, further boosting the performance of the regression model.