mlsanityData to 7 Oct 2026

CascadeXML: Rethinking Transformers for End-to-end Multi-resolution Training in Extreme Multi-label Classification

arXiv 2211.00640 ↗First posted 29 Oct 2022cs.LG cs.CL stat.MLcited by 12

Not on the map. Why?

Abstract (arXiv)

Extreme Multi-label Text Classification (XMC) involves learning a classifier that can assign an input with a subset of most relevant labels from millions of label choices. Recent approaches, such as XR-Transformer and LightXML, leverage a transformer instance to achieve state-of-the-art performance. However, in this process, these approaches need to make various trade-offs between performance and computational requirements. A major shortcoming, as compared to the Bi-LSTM based AttentionXML, is that they fail to keep separate feature representations for each resolution in a label tree. We thus propose CascadeXML, an end-to-end multi-resolution learning pipeline, which can harness the multi-layered architecture of a transformer model for attending to different label resolutions with separate feature representations. CascadeXML significantly outperforms all existing approaches with non-trivial gains obtained on benchmark datasets consisting of up to three million labels. Code for CascadeXML will be made publicly available at urlhttps://github.com/xmc-aalto/cascadexml.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 31 of its 57 references that are in the map. First posted on arXiv 29 Oct 2022.

Builds on

31 of 57 references.

References · 31

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  5. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,967
  6. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,801
  7. 2014Convolutional Neural Networks for Sentence Classificationcited by 1,124
  8. 2019Latent Retrieval for Weakly Supervised Open Domain Question Answeringcited by 691
  9. 2017Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolutioncited by 452
  10. 2020DeeBERT: Dynamic Early Exiting for Accelerating BERT Inferencecited by 322
  11. 2018Insights on representational similarity in neural networks with canonical correlationcited by 318
  12. 2020Pre-training Tasks for Embedding-based Large-scale Retrievalcited by 153
19 more
  1. 2018AttentionXML: Label Tree-based Attention-Aware Deep Model for High-Performance Extreme Multi-Label Text Classificationcited by 101
  2. 2016DiSMEC - Distributed Sparse Machines for Extreme Multi-label Classificationcited by 70
  3. 2021Fast Multi-Resolution Transformer Fine-tuning for Extreme Multi-label Text Classificationcited by 66
  4. 2019Bonsai -- Diverse and Shallow Trees for Extreme Multi-label Classificationcited by 61
  5. 2021LightXML: Transformer with Dynamic Negative Sampling for High-Performance Extreme Multi-label Text Classificationcited by 61
  6. 2018A no-regret generalization of hierarchical softmax to extreme multi-label classificationcited by 52
  7. 2020PECOS: Prediction for Enormous and Correlated Output Spacescited by 50
  8. 2021ECLARE: Extreme Classification with Label Graph Correlationscited by 38
  9. 2021DeepXML: A Deep Extreme Multi-Label Learning Framework Applied to Short Text Documentscited by 37
  10. 2021DECAF: Deep Extreme Classification with Label Featurescited by 33
  11. 2018Stochastic Negative Mining for Learning with Large Output Spacescited by 29
  12. 2021Extreme Multi-label Learning for Semantic Matching in Product Searchcited by 28
  13. 2020Pretrained Generalized Autoregressive Model with Adaptive Probabilistic Label Clusters for Extreme Multi-label Text Classificationcited by 25
  14. 2021Extreme Zero-Shot Learning for Extreme Text Classificationcited by 15
  15. 2022On Missing Labels, Long-tails and Propensities in Extreme Multi-label Classificationcited by 15
  16. 2022Metadata-Induced Contrastive Learning for Zero-Shot Multi-Label Text Classificationcited by 10
  17. 2021Propensity-scored Probabilistic Label Treescited by 8
  18. 2019LdSM: Logarithm-depth Streaming Multi-label Decision Treescited by 3
  19. 2021Speeding-up One-vs-All Training for Extreme Classification via Smart Initializationcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 5; 2025: 1; 2026: 3.

12 papers.

  1. 2023Generalized test utilities for long-tail performance in extreme multi-label classificationcited by 6
  2. 2023Towards Memory-Efficient Training for Extremely Large Output Spaces -- Learning with 500k Labels on a Single Commodity GPUcited by 5
  3. 2025DynaSpec: Context-aware Dynamic Speculative Sampling for Large-Vocabulary Language Modelscited by 4
  4. 2023MatchXML: An Efficient Text-label Matching Framework for Extreme Multi-label Text Classificationcited by 3
  5. 2024Navigating Extremes: Dynamic Sparsity in Large Output Spacescited by 3
  6. 2024Graph Regularized Encoder Training for Extreme Classificationcited by 1
  7. 2024UniDEC : Unified Dual Encoder and Classifier Training for Extreme Multi-Label Classificationcited by 1
  8. 2024PerOS: Personalized Self-Adapting Operating Systems in the Cloudcited by 0
  9. 2024Don't Pay Attention, PLANT It: Pretraining Attention via Learning-to-Rankcited by 0
  10. 2026FEAST: Retrieval-Augmented Multi-Hierarchical Food Classification for the FoodEx2 Systemcited by 0
  11. 2026HASTE: Hardware-Aware Dynamic Sparse Training for Large Output Spacescited by 0
  12. 2026LLM-Enhanced Dual-Branch Learning for Large-Scale Multi-Label Text Classificationcited by 0