# Long-tailed Extreme Multi-label Text Classification with Generated Pseudo Label Descriptions

arXiv: https://arxiv.org/abs/2204.00958 · HTML: https://mlsanity.com/paper/2204.00958/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 35 references that are in the map.
- First posted on arXiv 2 Apr 2022.

## Builds on (most cited first, 14 of 14)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval](https://mlsanity.com/paper/2007.00808/) (2020)
- [Sparse, Dense, and Attentional Representations for Text Retrieval](https://mlsanity.com/paper/2005.00181/) (2020)
- [Unsupervised Corpus Aware Language Model Pre-training for Dense Passage Retrieval](https://mlsanity.com/paper/2108.05540/) (2021)
- [COIL: Revisit Exact Lexical Match in Information Retrieval with Contextualized Inverted List](https://mlsanity.com/paper/2104.07186/) (2021)
- [AttentionXML: Label Tree-based Attention-Aware Deep Model for High-Performance Extreme Multi-Label Text Classification](https://mlsanity.com/paper/1811.01727/) (2018)
- [DiSMEC - Distributed Sparse Machines for Extreme Multi-label Classification](https://mlsanity.com/paper/1609.02521/) (2016)
- [Bonsai -- Diverse and Shallow Trees for Extreme Multi-label Classification](https://mlsanity.com/paper/1904.08249/) (2019)
- [LightXML: Transformer with Dynamic Negative Sampling for High-Performance Extreme Multi-label Text Classification](https://mlsanity.com/paper/2101.03305/) (2021)
- [Description Based Text Classification with Reinforcement Learning](https://mlsanity.com/paper/2002.03067/) (2020)
- [Complementing Lexical Retrieval with Semantic Residual Embedding](https://mlsanity.com/paper/2004.13969/) (2020)
- [Pretrained Generalized Autoregressive Model with Adaptive Probabilistic Label Clusters for Extreme Multi-label Text Classification](https://mlsanity.com/paper/2007.02439/) (2020)
- [Contextualized Sparse Representations for Real-Time Open-Domain Question Answering](https://mlsanity.com/paper/1911.02896/) (2019)

## Built on by (most cited first, 5 of 5)

- [Generalized test utilities for long-tail performance in extreme multi-label classification](https://mlsanity.com/paper/2311.05081/) (2023)
- [Description-Enhanced Label Embedding Contrastive Learning for Text Classification](https://mlsanity.com/paper/2306.08817/) (2023)
- [Adaptive Thresholding for Multi-Label Classification via Global-Local Signal Fusion](https://mlsanity.com/paper/2505.03118/) (2025)
- [Cross Encoding as Augmentation: Towards Effective Educational Text Classification](https://mlsanity.com/paper/2305.18977/) (2023)
- [CL-UZH at SemEval-2023 Task 10: Sexism Detection through Incremental Fine-Tuning and Multi-Task Learning with Label Descriptions](https://mlsanity.com/paper/2306.03907/) (2023)

## Abstract (arXiv)

Extreme Multi-label Text Classification (XMTC) has been a tough challenge in machine learning research and applications due to the sheer sizes of the label spaces and the severe data scarce problem associated with the long tail of rare labels in highly skewed distributions. This paper addresses the challenge of tail label prediction by proposing a novel approach, which combines the effectiveness of a trained bag-of-words (BoW) classifier in generating informative label descriptions under severe data scarce conditions, and the power of neural embedding based retrieval models in mapping input documents (as queries) to relevant label descriptions. The proposed approach achieves state-of-the-art performance on XMTC benchmark datasets and significantly outperforms the best methods so far in the tail label prediction. We also provide a theoretical analysis for relating the BoW and neural models w.r.t. performance lower bound.