# Loss Functions for Classification using Structured Entropy

arXiv: https://arxiv.org/abs/2206.07122 · HTML: https://mlsanity.com/paper/2206.07122/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 39 references that are in the map.
- First posted on arXiv 14 Jun 2022.

## Builds on (most cited first, 17 of 17)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distances](https://mlsanity.com/paper/1306.0895/) (2013)
- [Contrastive Multiview Coding](https://mlsanity.com/paper/1906.05849/) (2019)
- [Learning Generative Models with Sinkhorn Divergences](https://mlsanity.com/paper/1706.00292/) (2017)
- [Fast Computation of Wasserstein Barycenters](https://mlsanity.com/paper/1310.4375/) (2013)
- [Learning with a Wasserstein Loss](https://mlsanity.com/paper/1506.05439/) (2015)
- [Making Better Mistakes: Leveraging Class Hierarchies with Deep Networks](https://mlsanity.com/paper/1912.09393/) (2019)
- [Integrating domain knowledge: using hierarchies to improve deep classifiers](https://mlsanity.com/paper/1811.07125/) (2018)
- [A hierarchical loss and its problems when classifying non-hierarchically](https://mlsanity.com/paper/1709.01062/) (2017)
- [Understanding the Impact of Label Granularity on CNN-based Image Classification](https://mlsanity.com/paper/1901.07012/) (2019)
- [GAIT: A Geometric Approach to Information Theory](https://mlsanity.com/paper/1906.08325/) (2019)
- [Compact Multi-Class Boosted Trees](https://mlsanity.com/paper/1710.11547/) (2017)
- [Probabilistic Cascading for Large Scale Hierarchical Classification](https://mlsanity.com/paper/1505.02251/) (2015)
- [Exploiting Categorical Structure Using Tree-Based Methods](https://mlsanity.com/paper/2004.07383/) (2020)

## Built on by (most cited first, 1 of 1)

- [Nonparametric Probabilistic Regression with Coarse Learners](https://mlsanity.com/paper/2210.16247/) (2022)

## Abstract (arXiv)

Cross-entropy loss is the standard metric used to train classification models in deep learning and gradient boosting. It is well-known that this loss function fails to account for similarities between the different values of the target. We propose a generalization of entropy called em structured entropy which uses a random partition to incorporate the structure of the target variable in a manner which retains many theoretical properties of standard entropy. We show that a structured cross-entropy loss yields better results on several classification problems where the target variable has an a priori known structure. The approach is simple, flexible, easily computable, and does not rely on a hierarchically defined notion of structure.