# PECOS: Prediction for Enormous and Correlated Output Spaces

arXiv: https://arxiv.org/abs/2010.05878 · HTML: https://mlsanity.com/paper/2010.05878/

## Key facts

- Cited by 50 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Oct 2020.

## Built on by (most cited first, 30 of 50)

- [Matryoshka Representation Learning](https://mlsanity.com/paper/2205.13147/) (2022)
- [Node Feature Extraction by Self-Supervised Multi-scale Neighborhood Prediction](https://mlsanity.com/paper/2111.00064/) (2021)
- [Fast Multi-Resolution Transformer Fine-tuning for Extreme Multi-label Text Classification](https://mlsanity.com/paper/2110.00685/) (2021)
- [Extreme Multi-label Learning for Semantic Matching in Product Search](https://mlsanity.com/paper/2106.12657/) (2021)
- [Deep Hierarchy in Bandits](https://mlsanity.com/paper/2202.01454/) (2022)
- [The Effect of Metadata on Scientific Literature Tagging: A Cross-Field Cross-Model Study](https://mlsanity.com/paper/2302.03341/) (2023)
- [Extreme Zero-Shot Learning for Extreme Text Classification](https://mlsanity.com/paper/2112.08652/) (2021)
- [ELIAS: End-to-End Learning to Index and Search in Large Output Spaces](https://mlsanity.com/paper/2210.08410/) (2022)
- [CascadeXML: Rethinking Transformers for End-to-end Multi-resolution Training in Extreme Multi-label Classification](https://mlsanity.com/paper/2211.00640/) (2022)
- [Taming the Sigmoid Bottleneck: Provably Argmaxable Sparse Multi-Label Classification](https://mlsanity.com/paper/2310.10443/) (2023)
- [SCR: Training Graph Neural Networks with Consistency Regularization](https://mlsanity.com/paper/2112.04319/) (2021)
- [SemEval-2025 Task 5: LLMs4Subjects -- LLM-based Automated Subject Tagging for a National Technical Library's Open-Access Catalog](https://mlsanity.com/paper/2504.07199/) (2025)
- [Top-k eXtreme Contextual Bandits with Arm Hierarchy](https://mlsanity.com/paper/2102.07800/) (2021)
- [InceptionXML: A Lightweight Framework with Synchronized Negative Sampling for Short Text Extreme Classification](https://mlsanity.com/paper/2109.07319/) (2021)
- [Propensity-scored Probabilistic Label Trees](https://mlsanity.com/paper/2110.10803/) (2021)
- [Label Disentanglement in Partition-based Extreme Multilabel Classification](https://mlsanity.com/paper/2106.12751/) (2021)
- [A Survey on Extreme Multi-label Learning](https://mlsanity.com/paper/2210.03968/) (2022)
- [PINA: Leveraging Side Information in eXtreme Multi-label Classification via Predicted Instance Neighborhood Aggregation](https://mlsanity.com/paper/2305.12349/) (2023)
- [Linear Classifier: An Often-Forgotten Baseline for Text Classification](https://mlsanity.com/paper/2306.07111/) (2023)
- [Dual-Encoders for Extreme Multi-Label Classification](https://mlsanity.com/paper/2310.10636/) (2023)
- [Towards Heterogeneous Long-tailed Learning: Benchmarking, Metrics, and Toolbox](https://mlsanity.com/paper/2307.08235/) (2023)
- [Improving Dual-Encoder Training through Dynamic Indexes for Negative Mining](https://mlsanity.com/paper/2303.15311/) (2023)
- [FINGER: Fast Inference for Graph-based Approximate Nearest Neighbor Search](https://mlsanity.com/paper/2206.11408/) (2022)
- [PEFA: Parameter-Free Adapters for Large-scale Embedding-based Retrieval Models](https://mlsanity.com/paper/2312.02429/) (2023)
- [Noise Contrastive Estimation-based Matching Framework for Low-Resource Security Attack Pattern Recognition](https://mlsanity.com/paper/2401.10337/) (2024)
- [Annif at SemEval-2025 Task 5: Traditional XMTC augmented by LLMs](https://mlsanity.com/paper/2504.19675/) (2025)
- [LLM-guided Hierarchical Search for End-to-end Reasoning Intensive Retrieval](https://mlsanity.com/paper/2510.13217/) (2025)
- [MatchXML: An Efficient Text-label Matching Framework for Extreme Multi-label Text Classification](https://mlsanity.com/paper/2308.13139/) (2023)
- [Enhancing Group Fairness in Online Settings Using Oblique Decision Forests](https://mlsanity.com/paper/2310.11401/) (2023)
- [Dense Retrieval as Indirect Supervision for Large-space Decision Making](https://mlsanity.com/paper/2310.18619/) (2023)

## Abstract (arXiv)

Many large-scale applications amount to finding relevant results from an enormous output space of potential candidates. For example, finding the best matching product from a large catalog or suggesting related search phrases on a search engine. The size of the output space for these problems can range from millions to billions, and can even be infinite in some applications. Moreover, training data is often limited for the long-tail items in the output space. Fortunately, items in the output space are often correlated thereby presenting an opportunity to alleviate the data sparsity issue. In this paper, we propose the Prediction for Enormous and Correlated Output Spaces (PECOS) framework, a versatile and modular machine learning framework for solving prediction problems for very large output spaces, and apply it to the eXtreme Multilabel Ranking (XMR) problem: given an input instance, find and rank the most relevant items from an enormous but fixed and finite output space. We propose a three phase framework for PECOS: (i) in the first phase, PECOS organizes the output space using a semantic indexing scheme, (ii) in the second phase, PECOS uses the indexing to narrow down the output space by orders of magnitude using a machine learned matching scheme, and (iii) in the third phase, PECOS ranks the matched items using a final ranking scheme. The versatility and modularity of PECOS allows for easy plug-and-play of various choices for the indexing, matching, and ranking phases. We also develop very fast inference procedures which allow us to perform XMR predictions in real time; for example, inference takes less than 1 millisecond per input on the dataset with 2.8 million labels. The PECOS software is available at https://libpecos.org.