# On Clustering Categories of Categorical Predictors in Generalized Linear Models

arXiv: https://arxiv.org/abs/2110.10059 · HTML: https://mlsanity.com/paper/2110.10059/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 17 references that are in the map.
- First posted on arXiv 19 Oct 2021.

## Builds on (most cited first, 5 of 5)

- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Similarity encoding for learning with dirty categorical variables](https://mlsanity.com/paper/1806.00979/) (2018)
- [Explaining Classification Models Built on High-Dimensional Sparse Data](https://mlsanity.com/paper/1607.06280/) (2016)
- [Sufficient Representations for Categorical Variables](https://mlsanity.com/paper/1908.09874/) (2019)
- [A Note on Coding and Standardization of Categorical Variables in (Sparse) Group Lasso Regression](https://mlsanity.com/paper/1805.06915/) (2018)

## Built on by (most cited first, 1 of 1)

- [Categorical data clustering: 25 years beyond K-modes](https://mlsanity.com/paper/2408.17244/) (2024)

## Abstract (arXiv)

We propose a method to reduce the complexity of Generalized Linear Models in the presence of categorical predictors. The traditional one-hot encoding, where each category is represented by a dummy variable, can be wasteful, difficult to interpret, and prone to overfitting, especially when dealing with high-cardinality categorical predictors. This paper addresses these challenges by finding a reduced representation of the categorical predictors by clustering their categories. This is done through a numerical method which aims to preserve (or even, improve) accuracy, while reducing the number of coefficients to be estimated for the categorical predictors. Thanks to its design, we are able to derive a proximity measure between categories of a categorical predictor that can be easily visualized. We illustrate the performance of our approach in real-world classification and count-data datasets where we see that clustering the categorical predictors reduces complexity substantially without harming accuracy.