# Encoding high-cardinality string categorical variables

arXiv: https://arxiv.org/abs/1907.01860 · HTML: https://mlsanity.com/paper/1907.01860/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Jul 2019.

## Built on by (most cited first, 23 of 23)

- [CARTE: Pretraining and Transfer for Tabular Learning](https://mlsanity.com/paper/2402.16785/) (2024)
- [Regularized target encoding outperforms traditional methods in supervised machine learning with high cardinality features](https://mlsanity.com/paper/2104.00629/) (2021)
- [Table Foundation Models: on knowledge pre-training for tabular learning](https://mlsanity.com/paper/2505.14415/) (2025)
- [TabSTAR: A Tabular Foundation Model for Tabular Data with Text Fields](https://mlsanity.com/paper/2505.18125/) (2025)
- [Automated data processing and feature engineering for deep learning and big data applications: a survey](https://mlsanity.com/paper/2403.11395/) (2024)
- [Vectorizing string entries for data processing on tables: when are larger language models better?](https://mlsanity.com/paper/2312.09634/) (2023)
- [Fairness Implications of Encoding Protected Categorical Attributes](https://mlsanity.com/paper/2201.11358/) (2022)
- [A benchmark of categorical encoders for binary classification](https://mlsanity.com/paper/2307.09191/) (2023)
- [Encoding categorical data: Is there yet anything 'hotter' than one-hot encoding?](https://mlsanity.com/paper/2312.16930/) (2023)
- [Interpretable vs Learned Encoders for High-Cardinality Fraud Detection](https://mlsanity.com/paper/2607.00477/) (2026)
- [Comparative Study on the Performance of Categorical Variable Encoders in Classification and Regression Tasks](https://mlsanity.com/paper/2401.09682/) (2024)
- [Counterfactual Situation Testing: From Single to Multidimensional Discrimination](https://mlsanity.com/paper/2502.01267/) (2025)
- [Predicting Treatment Adherence of Tuberculosis Patients at Scale](https://mlsanity.com/paper/2211.02943/) (2022)
- [From Strings to Data Science: a Practical Framework for Automated String Handling](https://mlsanity.com/paper/2111.01868/) (2021)
- [URANUS: Radio Frequency Tracking, Classification and Identification of Unmanned Aircraft Vehicles](https://mlsanity.com/paper/2207.06025/) (2022)
- [Progressive Feature Upgrade in Semi-supervised Learning on Tabular Domain](https://mlsanity.com/paper/2212.00892/) (2022)
- [Categorising Products in an Online Marketplace: An Ensemble Approach](https://mlsanity.com/paper/2304.13852/) (2023)
- [Predicting delays in Indian lower courts using AutoML and Decision Forests](https://mlsanity.com/paper/2307.16285/) (2023)
- [CAVIAR: Categorical-Variable Embeddings for Accurate and Robust Inference](https://mlsanity.com/paper/2404.04979/) (2024)
- [CLAMS: A System for Zero-Shot Model Selection for Clustering](https://mlsanity.com/paper/2407.11286/) (2024)
- [Robustness of AutoML on Dirty Categorical Data](https://mlsanity.com/paper/2602.00412/) (2026)
- [Enhancing the KidSat Model: Integrating Geographical Encoding and Data Quality Assessment for Childhood Poverty Prediction](https://mlsanity.com/paper/2607.08281/) (2026)
- [TILE: Bridging LLMs and Table Foundation Models for Tabular Learning with Strings](https://mlsanity.com/paper/2610.17065/) (2026)

## Abstract (arXiv)

Statistical models usually require vector representations of categorical variables, using for instance one-hot encoding. This strategy breaks down when the number of categories grows, as it creates high-dimensional feature vectors. Additionally, for string entries, one-hot encoding does not capture information in their representation.Here, we seek low-dimensional encoding of high-cardinality string categorical variables. Ideally, these should be: scalable to many categories; interpretable to end users; and facilitate statistical analysis. We introduce two encoding approaches for string categories: a Gamma-Poisson matrix factorization on substring counts, and the min-hash encoder, for fast approximation of string similarities. We show that min-hash turns set inclusions into inequality relations that are easier to learn. Both approaches are scalable and streamable. Experiments on real and simulated data show that these methods improve supervised learning with high-cardinality categorical variables. We recommend the following: if scalability is central, the min-hash encoder is the best option as it does not require any data fit; if interpretability is important, the Gamma-Poisson factorization is the best alternative, as it can be interpreted as one-hot encoding on inferred categories with informative feature names. Both models enable autoML on the original string entries as they remove the need for feature engineering or data cleaning.