# Pretrained Generalized Autoregressive Model with Adaptive Probabilistic Label Clusters for Extreme Multi-label Text Classification

arXiv: https://arxiv.org/abs/2007.02439 · HTML: https://mlsanity.com/paper/2007.02439/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Jul 2020.

## Built on by (most cited first, 25 of 25)

- [Node Feature Extraction by Self-Supervised Multi-scale Neighborhood Prediction](https://mlsanity.com/paper/2111.00064/) (2021)
- [Fast Multi-Resolution Transformer Fine-tuning for Extreme Multi-label Text Classification](https://mlsanity.com/paper/2110.00685/) (2021)
- [DeepXML: A Deep Extreme Multi-Label Learning Framework Applied to Short Text Documents](https://mlsanity.com/paper/2111.06685/) (2021)
- [NGAME: Negative Mining-aware Mini-batching for Extreme Classification](https://mlsanity.com/paper/2207.04452/) (2022)
- [The Effect of Metadata on Scientific Literature Tagging: A Cross-Field Cross-Model Study](https://mlsanity.com/paper/2302.03341/) (2023)
- [CascadeXML: Rethinking Transformers for End-to-end Multi-resolution Training in Extreme Multi-label Classification](https://mlsanity.com/paper/2211.00640/) (2022)
- [InceptionXML: A Lightweight Framework with Synchronized Negative Sampling for Short Text Extreme Classification](https://mlsanity.com/paper/2109.07319/) (2021)
- [A Survey on Extreme Multi-label Learning](https://mlsanity.com/paper/2210.03968/) (2022)
- [PINA: Leveraging Side Information in eXtreme Multi-label Classification via Predicted Instance Neighborhood Aggregation](https://mlsanity.com/paper/2305.12349/) (2023)
- [Dual-Encoders for Extreme Multi-Label Classification](https://mlsanity.com/paper/2310.10636/) (2023)
- [Generalized test utilities for long-tail performance in extreme multi-label classification](https://mlsanity.com/paper/2311.05081/) (2023)
- [Long-tailed Extreme Multi-label Text Classification with Generated Pseudo Label Descriptions](https://mlsanity.com/paper/2204.00958/) (2022)
- [MatchXML: An Efficient Text-label Matching Framework for Extreme Multi-label Text Classification](https://mlsanity.com/paper/2308.13139/) (2023)
- [Learning label-label correlations in Extreme Multi-label Classification via Label Features](https://mlsanity.com/paper/2405.04545/) (2024)
- [Adversarial Examples for Extreme Multilabel Text Classification](https://mlsanity.com/paper/2112.07512/) (2021)
- [Light-weight Deep Extreme Multilabel Classification](https://mlsanity.com/paper/2304.11045/) (2023)
- [GUDN: A novel guide network with label reinforcement strategy for extreme multi-label text classification](https://mlsanity.com/paper/2201.11582/) (2022)
- [Cluster-Guided Label Generation in Extreme Multi-Label Classification](https://mlsanity.com/paper/2302.09150/) (2023)
- [The Right Model for the Job: An Evaluation of Legal Multi-Label Classification Baselines](https://mlsanity.com/paper/2401.11852/) (2024)
- [Graph Regularized Encoder Training for Extreme Classification](https://mlsanity.com/paper/2402.18434/) (2024)
- [Multi-label Learning with Random Circular Vectors](https://mlsanity.com/paper/2407.05656/) (2024)
- [Exploiting Local and Global Features in Transformer-based Extreme Multi-label Text Classification](https://mlsanity.com/paper/2204.00933/) (2022)
- [PETA: Evaluating the Impact of Protein Transfer Learning with Sub-word Tokenization on Downstream Applications](https://mlsanity.com/paper/2310.17415/) (2023)
- [On the Necessity of World Knowledge for Mitigating Missing Labels in Extreme Classification](https://mlsanity.com/paper/2408.09585/) (2024)
- [LLM-Enhanced Dual-Branch Learning for Large-Scale Multi-Label Text Classification](https://mlsanity.com/paper/2609.12915/) (2026)

## Abstract (arXiv)

Extreme multi-label text classification (XMTC) is a task for tagging a given text with the most relevant labels from an extremely large label set. We propose a novel deep learning method called APLC-XLNet. Our approach fine-tunes the recently released generalized autoregressive pretrained model (XLNet) to learn a dense representation for the input text. We propose Adaptive Probabilistic Label Clusters (APLC) to approximate the cross entropy loss by exploiting the unbalanced label distribution to form clusters that explicitly reduce the computational time. Our experiments, carried out on five benchmark datasets, show that our approach has achieved new state-of-the-art results on four benchmark datasets. Our source code is available publicly at https://github.com/huiyegit/APLC_XLNet.