# Effectiveness of Hierarchical Softmax in Large Scale Classification Tasks

arXiv: https://arxiv.org/abs/1812.05737 · HTML: https://mlsanity.com/paper/1812.05737/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Dec 2018.

## Built on by (most cited first, 8 of 8)

- [Hierarchical Softmax for End-to-End Low-resource Multilingual Speech Recognition](https://mlsanity.com/paper/2204.03855/) (2022)
- [Distributionally Robust Classification on a Data Budget](https://mlsanity.com/paper/2308.03821/) (2023)
- [Predictive Representation Learning for Language Modeling](https://mlsanity.com/paper/2105.14214/) (2021)
- [Multi-View Stenosis Classification Leveraging Transformer-Based Multiple-Instance Learning Using Real-World Clinical Data](https://mlsanity.com/paper/2602.02067/) (2026)
- [The Tree Loss: Improving Generalization with Many Classes](https://mlsanity.com/paper/2204.07727/) (2022)
- [TypoSwype: An Imaging Approach to Detect Typo-Squatting](https://mlsanity.com/paper/2209.00783/) (2022)
- [Indeterminate Probability Theory](https://mlsanity.com/paper/2303.11536/) (2023)
- [Cross-lingual Embedding Clustering for Hierarchical Softmax in Low-Resource Multilingual Speech Recognition](https://mlsanity.com/paper/2501.17615/) (2025)

## Abstract (arXiv)

Typically, Softmax is used in the final layer of a neural network to get a probability distribution for output classes. But the main problem with Softmax is that it is computationally expensive for large scale data sets with large number of possible outputs. To approximate class probability efficiently on such large scale data sets we can use Hierarchical Softmax. LSHTC datasets were used to study the performance of the Hierarchical Softmax. LSHTC datasets have large number of categories. In this paper we evaluate and report the performance of normal Softmax Vs Hierarchical Softmax on LSHTC datasets. This evaluation used macro f1 score as a performance measure. The observation was that the performance of Hierarchical Softmax degrades as the number of classes increase.