# Beyond One-hot Encoding: lower dimensional target embedding

arXiv: https://arxiv.org/abs/1806.10805 · HTML: https://mlsanity.com/paper/1806.10805/

## Key facts

- Cited by 43 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Jun 2018.

## Built on by (most cited first, 30 of 43)

- [IndexTTS2: A Breakthrough in Emotionally Expressive and Duration-Controlled Auto-Regressive Zero-Shot Text-to-Speech](https://mlsanity.com/paper/2506.21619/) (2025)
- [Regularized target encoding outperforms traditional methods in supervised machine learning with high cardinality features](https://mlsanity.com/paper/2104.00629/) (2021)
- [Permutation Equivariant Graph Framelets for Heterophilous Graph Learning](https://mlsanity.com/paper/2306.04265/) (2023)
- [A Comprehensive Benchmark of Machine and Deep Learning Across Diverse Tabular Datasets](https://mlsanity.com/paper/2408.14817/) (2024)
- [SecGNN: Privacy-Preserving Graph Neural Network Training and Inference as a Cloud Service](https://mlsanity.com/paper/2202.07835/) (2022)
- [Looper: An end-to-end ML platform for product decisions](https://mlsanity.com/paper/2110.07554/) (2021)
- [Fairness Implications of Encoding Protected Categorical Attributes](https://mlsanity.com/paper/2201.11358/) (2022)
- [BiFeat: Supercharge GNN Training via Graph Feature Quantization](https://mlsanity.com/paper/2207.14696/) (2022)
- [WaveStitch: Flexible and Fast Conditional Time Series Generation with Diffusion Models](https://mlsanity.com/paper/2503.06231/) (2025)
- [No Query, No Access](https://mlsanity.com/paper/2505.07258/) (2025)
- [Temporal-spatial Correlation Attention Network for Clinical Data Analysis in Intensive Care Unit](https://mlsanity.com/paper/2306.01970/) (2023)
- [Hybrid Machine Learning techniques in the management of harmful algal blooms impact](https://mlsanity.com/paper/2402.09271/) (2024)
- [End-to-End Learning for Uplink MU-SIMO Joint Transmitter and Non-Coherent Receiver Design in Fading Channels](https://mlsanity.com/paper/2105.01260/) (2021)
- [The Role of Codeword-to-Class Assignments in Error-Correcting Codes: An Empirical Study](https://mlsanity.com/paper/2302.05334/) (2023)
- [Integrating Edges into U-Net Models with Explainable Activation Maps for Brain Tumor Segmentation using MR Images](https://mlsanity.com/paper/2401.01303/) (2024)
- [Exploring the Advantages of Dense-Vector to One-Hot Encoding of Intent Classes in Out-of-Scope Detection Tasks](https://mlsanity.com/paper/2205.09021/) (2022)
- [T-Explainer: A Model-Agnostic Explainability Framework Based on Gradients](https://mlsanity.com/paper/2404.16495/) (2024)
- [What Time Tells Us? An Explorative Study of Time Awareness Learned from Static Images](https://mlsanity.com/paper/2503.17899/) (2025)
- [Multi-task Adversarial Attacks against Black-box Model with Few-shot Queries](https://mlsanity.com/paper/2508.10039/) (2025)
- [PairConnect: A Compute-Efficient MLP Alternative to Attention](https://mlsanity.com/paper/2106.08235/) (2021)
- [WisdomNet: Prognosis of COVID-19 with Slender Prospect of False Negative Cases and Vaticinating the Probability of Maturation to ARDS using Posteroanterior Chest X-Rays](https://mlsanity.com/paper/2107.01392/) (2021)
- [Polarized skylight orientation determination artificial neural network](https://mlsanity.com/paper/2107.02328/) (2021)
- [Predicting the Travel Distance of Patients to Access Healthcare using Deep Neural Networks](https://mlsanity.com/paper/2112.03541/) (2021)
- [Cross-Media Scientific Research Achievements Retrieval Based on Deep Language Model](https://mlsanity.com/paper/2203.15595/) (2022)
- [U(1) Symmetry-breaking Observed in Generic CNN Bottleneck Layers](https://mlsanity.com/paper/2206.02220/) (2022)
- [URANUS: Radio Frequency Tracking, Classification and Identification of Unmanned Aircraft Vehicles](https://mlsanity.com/paper/2207.06025/) (2022)
- [Constraining Representations Yields Models That Know What They Don't Know](https://mlsanity.com/paper/2208.14488/) (2022)
- [CogReact: A Reinforced Framework to Model Human Cognitive Reaction Modulated by Dynamic Intervention](https://mlsanity.com/paper/2301.06216/) (2023)
- [Label Embedding via Low-Coherence Matrices](https://mlsanity.com/paper/2305.19470/) (2023)
- [Machine Learning Approach on Multiclass Classification of Internet Firewall Log Files](https://mlsanity.com/paper/2306.07997/) (2023)

## Abstract (arXiv)

Target encoding plays a central role when learning Convolutional Neural Networks. In this realm, One-hot encoding is the most prevalent strategy due to its simplicity. However, this so widespread encoding schema assumes a flat label space, thus ignoring rich relationships existing among labels that can be exploited during training. In large-scale datasets, data does not span the full label space, but instead lies in a low-dimensional output manifold. Following this observation, we embed the targets into a low-dimensional space, drastically improving convergence speed while preserving accuracy. Our contribution is two fold: (i) We show that random projections of the label space are a valid tool to find such lower dimensional embeddings, boosting dramatically convergence rates at zero computational cost; and (ii) we propose a normalized eigenrepresentation of the class manifold that encodes the targets with minimal information loss, improving the accuracy of random projections encoding while enjoying the same convergence rates. Experiments on CIFAR-100, CUB200-2011, Imagenet, and MIT Places demonstrate that the proposed approach drastically improves convergence speed while reaching very competitive accuracy rates.