# Sufficient Representations for Categorical Variables

arXiv: https://arxiv.org/abs/1908.09874 · HTML: https://mlsanity.com/paper/1908.09874/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Aug 2019.

## Built on by (most cited first, 3 of 3)

- [Program Evaluation with Remotely Sensed Outcomes](https://mlsanity.com/paper/2411.10959/) (2024)
- [On Clustering Categories of Categorical Predictors in Generalized Linear Models](https://mlsanity.com/paper/2110.10059/) (2021)
- [Let Time Tell: Identification and Gaussian Process Estimation for Interrupted Time Series](https://mlsanity.com/paper/2608.20610/) (2026)

## Abstract (arXiv)

Many learning algorithms require categorical data to be transformed into real vectors before it can be used as input. Often, categorical variables are encoded as one-hot (or dummy) vectors. However, this mode of representation can be wasteful since it adds many low-signal regressors, especially when the number of unique categories is large. In this paper, we investigate simple alternative solutions for universally consistent estimators that rely on lower-dimensional real-valued representations of categorical variables that are "sufficient" in the sense that no predictive information is lost. We then compare preexisting and proposed methods on simulated and observational datasets.