# Efficient Representations for High-Cardinality Categorical Variables in Machine Learning

arXiv: https://arxiv.org/abs/2501.05646 · HTML: https://mlsanity.com/paper/2501.05646/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 2 of its 18 references that are in the map.
- First posted on arXiv 10 Jan 2025.

## Builds on (most cited first, 2 of 2)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)

## Built on by (most cited first, 10 of 10)

- [What's on My Network? Using Large Language Models to Identify Real-World IoT Devices at Scale](https://mlsanity.com/paper/2510.13817/) (2025)
- [GMA-SAWGAN-GP: A Novel Data Generative Framework to Enhance IDS Detection Performance](https://mlsanity.com/paper/2603.28838/) (2026)
- [MMbeddings: Parameter-Efficient, Low-Overfitting Probabilistic Embeddings Inspired by Nonlinear Mixed Models](https://mlsanity.com/paper/2510.22198/) (2025)
- [Evaluating Tabular Representation Learning for Network Intrusion Detection](https://mlsanity.com/paper/2605.02519/) (2026)
- [Feature-Optimized Vision for Adaptive 3D Scene Reconstruction](https://mlsanity.com/paper/2605.31534/) (2026)
- [SPECTRA: Synthetic IR Test Collections with Relevance Oracles and Controlled Distractor Diagnostics](https://mlsanity.com/paper/2605.31575/) (2026)
- [SEMBridge: Tagless-Final Program Semantics with Weakest-Precondition and Bounded-Checking Interpretations](https://mlsanity.com/paper/2606.00220/) (2026)
- [SECUREVENT: Hybrid AI/ML Security Monitoring for Distributed Event-Based Systems](https://mlsanity.com/paper/2606.01741/) (2026)
- [Acceptance-Test-Driven Evaluation Protocols for Business-Centric LLM Systems](https://mlsanity.com/paper/2606.02755/) (2026)
- [Anycast Performance in Context](https://mlsanity.com/paper/2606.04298/) (2026)

## Abstract (arXiv)

High-cardinality categorical variables pose significant challenges in machine learning, particularly in terms of computational efficiency and model interpretability. Traditional one-hot encoding often results in high-dimensional sparse feature spaces, increasing the risk of overfitting and reducing scalability. This paper introduces novel encoding techniques, including means encoding, low-rank encoding, and multinomial logistic regression encoding, to address these challenges. These methods leverage sufficient representations to generate compact and informative embeddings of categorical data. We conduct rigorous theoretical analyses and empirical validations on diverse datasets, demonstrating significant improvements in model performance and computational efficiency compared to baseline methods. The proposed techniques are particularly effective in domains requiring scalable solutions for large datasets, paving the way for more robust and efficient applications in machine learning.