# Entity Embeddings of Categorical Variables

arXiv: https://arxiv.org/abs/1604.06737 · HTML: https://mlsanity.com/paper/1604.06737/

## Key facts

- Cited by 87 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Apr 2016.

## Built on by (most cited first, 30 of 87)

- [Deep Neural Networks and Tabular Data: A Survey](https://mlsanity.com/paper/2110.01889/) (2021)
- [CLIP-Driven Universal Model for Organ Segmentation and Tumor Detection](https://mlsanity.com/paper/2301.00785/) (2023)
- [Large Language Models(LLMs) on Tabular Data: Prediction, Generation, and Understanding -- A Survey](https://mlsanity.com/paper/2402.17944/) (2024)
- [YAHPO Gym -- An Efficient Multi-Objective Multi-Fidelity Benchmark for Hyperparameter Optimization](https://mlsanity.com/paper/2109.03670/) (2021)
- [ProcessTransformer: Predictive Business Process Monitoring with Transformer Network](https://mlsanity.com/paper/2104.00721/) (2021)
- [Benchmarking Multimodal AutoML for Tabular Data with Text Fields](https://mlsanity.com/paper/2111.02705/) (2021)
- [Integrating Random Effects in Deep Neural Networks](https://mlsanity.com/paper/2206.03314/) (2022)
- [Regularized target encoding outperforms traditional methods in supervised machine learning with high cardinality features](https://mlsanity.com/paper/2104.00629/) (2021)
- [Quick-Tune: Quickly Learning Which Pretrained Model to Finetune and How](https://mlsanity.com/paper/2306.03828/) (2023)
- [Trading with the Momentum Transformer: An Intelligent and Interpretable Architecture](https://mlsanity.com/paper/2112.08534/) (2021)
- [User Response Prediction in Online Advertising](https://mlsanity.com/paper/2101.02342/) (2021)
- [A Comparative Study of Methods for Estimating Conditional Shapley Values and When to Use Them](https://mlsanity.com/paper/2305.09536/) (2023)
- [CTSyn: A Foundation Model for Cross Tabular Data Generation](https://mlsanity.com/paper/2406.04619/) (2024)
- [Enhancing variational quantum state diagonalization using reinforcement learning techniques](https://mlsanity.com/paper/2306.11086/) (2023)
- [Combining Discrete Choice Models and Neural Networks through Embeddings: Formulation, Interpretability and Performance](https://mlsanity.com/paper/2109.12042/) (2021)
- [An Attention-based ConvLSTM Autoencoder with Dynamic Thresholding for Unsupervised Anomaly Detection in Multivariate Time Series](https://mlsanity.com/paper/2201.09172/) (2022)
- [PVG at WASSA 2021: A Multi-Input, Multi-Task, Transformer-Based Architecture for Empathy and Distress Prediction](https://mlsanity.com/paper/2103.03296/) (2021)
- [A Fair Pricing Model via Adversarial Learning](https://mlsanity.com/paper/2202.12008/) (2022)
- [Machine Learning with High-Cardinality Categorical Features in Actuarial Applications](https://mlsanity.com/paper/2301.12710/) (2023)
- [Neural networks for insurance pricing with frequency and severity data: a benchmark study from data preprocessing to technical tariff](https://mlsanity.com/paper/2310.12671/) (2023)
- [Midwifery Learning and Forecasting: Predicting Content Demand with User-Generated Logs](https://mlsanity.com/paper/2107.02480/) (2021)
- [Muddling Label Regularization: Deep Learning for Tabular Datasets](https://mlsanity.com/paper/2106.04462/) (2021)
- [Predicting Time-to-conversion for Dementia of Alzheimer's Type using Multi-modal Deep Survival Analysis](https://mlsanity.com/paper/2205.01188/) (2022)
- [The Credibility Transformer](https://mlsanity.com/paper/2409.16653/) (2024)
- [TabRep: Training Tabular Diffusion Models with a Simple and Effective Continuous Representation](https://mlsanity.com/paper/2504.04798/) (2025)
- [Trainable Discrete Feature Embeddings for Variational Quantum Classifier](https://mlsanity.com/paper/2106.09415/) (2021)
- [Task Embedding Temporal Convolution Networks for Transfer Learning Problems in Renewable Power Time-Series Forecast](https://mlsanity.com/paper/2204.13908/) (2022)
- [Few-Shot Learning Patterns in Financial Time-Series for Trend-Following Strategies](https://mlsanity.com/paper/2310.10500/) (2023)
- [On Learning Representations for Tabular Data Distillation](https://mlsanity.com/paper/2501.13905/) (2025)
- [Unsupervised Anomaly Detection for Auditing Data and Impact of Categorical Encodings](https://mlsanity.com/paper/2210.14056/) (2022)

## Abstract (arXiv)

We map categorical variables in a function approximation problem into Euclidean spaces, which are the entity embeddings of the categorical variables. The mapping is learned by a neural network during the standard supervised training process. Entity embedding not only reduces memory usage and speeds up neural networks compared with one-hot encoding, but more importantly by mapping similar values close to each other in the embedding space it reveals the intrinsic properties of the categorical variables. We applied it successfully in a recent Kaggle competition and were able to reach the third position with relative simple features. We further demonstrate in this paper that entity embedding helps the neural network to generalize better when the data is sparse and statistics is unknown. Thus it is especially useful for datasets with lots of high cardinality features, where other methods tend to overfit. We also demonstrate that the embeddings obtained from the trained neural network boost the performance of all tested machine learning methods considerably when used as the input features instead. As entity embedding defines a distance measure for categorical variables it can be used for visualizing categorical data and for data clustering.