# Taming the Cross Entropy Loss

arXiv: https://arxiv.org/abs/1810.05075 · HTML: https://mlsanity.com/paper/1810.05075/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Oct 2018.

## Built on by (most cited first, 10 of 10)

- [To Be Forgotten or To Be Fair: Unveiling Fairness Implications of Machine Unlearning Methods](https://mlsanity.com/paper/2302.03350/) (2023)
- [Generative AI voting: fair collective choice is resilient to LLM biases and inconsistencies](https://mlsanity.com/paper/2406.11871/) (2024)
- [Finger-NestNet: Interpretable Fingerphoto Verification on Smartphone using Deep Nested Residual Network](https://mlsanity.com/paper/2212.05884/) (2022)
- [Adversary Aware Continual Learning](https://mlsanity.com/paper/2304.14483/) (2023)
- [BS-Mamba for Black-Soil Area Detection On the Qinghai-Tibetan Plateau](https://mlsanity.com/paper/2503.12495/) (2025)
- [Centralized vs Decentralized Federated Learning: A trade-off performance analysis](https://mlsanity.com/paper/2605.16089/) (2026)
- [A novel multi-scale loss function for classification problems in machine learning](https://mlsanity.com/paper/2106.02676/) (2021)
- [Mixing between the Cross Entropy and the Expectation Loss Terms](https://mlsanity.com/paper/2109.05635/) (2021)
- [BOTracle: A framework for Discriminating Bots and Humans](https://mlsanity.com/paper/2412.02266/) (2024)
- [A Transformer-in-Transformer Network Utilizing Knowledge Distillation for Image Recognition](https://mlsanity.com/paper/2502.16762/) (2025)

## Abstract (arXiv)

We present the Tamed Cross Entropy (TCE) loss function, a robust derivative of the standard Cross Entropy (CE) loss used in deep learning for classification tasks. However, unlike other robust losses, the TCE loss is designed to exhibit the same training properties than the CE loss in noiseless scenarios. Therefore, the TCE loss requires no modification on the training regime compared to the CE loss and, in consequence, can be applied in all applications where the CE loss is currently used. We evaluate the TCE loss using the ResNet architecture on four image datasets that we artificially contaminated with various levels of label noise. The TCE loss outperforms the CE loss in every tested scenario.