# Open Vocabulary Extreme Classification Using Generative Models

arXiv: https://arxiv.org/abs/2205.05812 · HTML: https://mlsanity.com/paper/2205.05812/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 31 references that are in the map.
- First posted on arXiv 12 May 2022.

## Builds on (most cited first, 16 of 16)

- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Bag of Tricks for Efficient Text Classification](https://mlsanity.com/paper/1607.01759/) (2016)
- [Retrieval Augmentation Reduces Hallucination in Conversation](https://mlsanity.com/paper/2104.07567/) (2021)
- [Autoregressive Entity Retrieval](https://mlsanity.com/paper/2010.00904/) (2020)
- [Recent Advances in Open Set Recognition: A Survey](https://mlsanity.com/paper/1811.08581/) (2018)
- [Order Matters: Sequence to sequence for sets](https://mlsanity.com/paper/1511.06391/) (2015)
- [Large-Scale Multi-Label Text Classification on EU Legislation](https://mlsanity.com/paper/1906.02192/) (2019)
- [AttentionXML: Label Tree-based Attention-Aware Deep Model for High-Performance Extreme Multi-Label Text Classification](https://mlsanity.com/paper/1811.01727/) (2018)
- [DiSMEC - Distributed Sparse Machines for Extreme Multi-label Classification](https://mlsanity.com/paper/1609.02521/) (2016)
- [Bonsai -- Diverse and Shallow Trees for Extreme Multi-label Classification](https://mlsanity.com/paper/1904.08249/) (2019)
- [Microblog Hashtag Generation via Encoding Conversation Contexts](https://mlsanity.com/paper/1905.07584/) (2019)
- [Extreme Regression for Dynamic Search Advertising](https://mlsanity.com/paper/2001.05228/) (2020)
- [Unbiased Loss Functions for Extreme Classification With Missing Labels](https://mlsanity.com/paper/2007.00237/) (2020)
- [A Deep Reinforced Sequence-to-Set Model for Multi-Label Text Classification](https://mlsanity.com/paper/1809.03118/) (2018)

## Built on by (most cited first, 13 of 13)

- [A Survey on Extreme Multi-label Learning](https://mlsanity.com/paper/2210.03968/) (2022)
- [SemSup-XC: Semantic Supervision for Zero and Few-shot Extreme Classification](https://mlsanity.com/paper/2301.11309/) (2023)
- [ICXML: An In-Context Learning Framework for Zero-Shot Extreme Multi-Label Classification](https://mlsanity.com/paper/2311.09649/) (2023)
- [From Lazy to Prolific: Tackling Missing Labels in Open Vocabulary Extreme Classification by Positive-Unlabeled Sequence Learning](https://mlsanity.com/paper/2408.08981/) (2024)
- [Evaluating end-to-end entity linking on domain-specific knowledge bases: Learning about ancient technologies from museum collections](https://mlsanity.com/paper/2305.14588/) (2023)
- [GraphEx: A Graph-based Extraction Method for Advertiser Keyphrase Recommendation](https://mlsanity.com/paper/2409.03140/) (2024)
- [LLMDistill4Ads: Using Cross-Encoders to Distill from LLM Signals for Advertiser Keyphrase Recommendations at eBay](https://mlsanity.com/paper/2508.03628/) (2025)
- [Cluster-Guided Label Generation in Extreme Multi-Label Classification](https://mlsanity.com/paper/2302.09150/) (2023)
- [Prompt Tuned Embedding Classification for Multi-Label Industry Sector Allocation](https://mlsanity.com/paper/2309.12075/) (2023)
- [BroadGen: A Framework for Generating Effective and Efficient Advertiser Broad Match Keyphrase Recommendations](https://mlsanity.com/paper/2505.19164/) (2025)
- [Handling Open-Vocabulary Constructs in Formalizing Specifications: Retrieval-Augmented Parsing with Expert Knowledge](https://mlsanity.com/paper/2509.08808/) (2025)
- [Extreme Meta-Classification for Large-Scale Zero-Shot Retrieval](https://mlsanity.com/paper/2606.25237/) (2026)
- [MERIT: Mitigating Exposure Bias in Generative XMC for User-Interest Propensity Modeling](https://mlsanity.com/paper/2608.28931/) (2026)

## Abstract (arXiv)

The extreme multi-label classification (XMC) task aims at tagging content with a subset of labels from an extremely large label set. The label vocabulary is typically defined in advance by domain experts and assumed to capture all necessary tags. However in real world scenarios this label set, although large, is often incomplete and experts frequently need to refine it. To develop systems that simplify this process, we introduce the task of open vocabulary XMC (OXMC): given a piece of content, predict a set of labels, some of which may be outside of the known tag set. Hence, in addition to not having training data for some labels - as is the case in zero-shot classification - models need to invent some labels on-the-fly. We propose GROOV, a fine-tuned seq2seq model for OXMC that generates the set of labels as a flat sequence and is trained using a novel loss independent of predicted label order. We show the efficacy of the approach, experimenting with popular XMC datasets for which GROOV is able to predict meaningful labels outside the given vocabulary while performing on par with state-of-the-art solutions for known labels.