# In-Context Learning for Extreme Multi-Label Classification

arXiv: https://arxiv.org/abs/2401.12178 · HTML: https://mlsanity.com/paper/2401.12178/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 17 references that are in the map.
- First posted on arXiv 22 Jan 2024.

## Builds on (most cited first, 10 of 10)

- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines](https://mlsanity.com/paper/2310.03714/) (2023)
- [SkillSpan: Hard and Soft Skill Extraction from English Job Postings](https://mlsanity.com/paper/2204.12811/) (2022)
- [IDAS: Intent Discovery with Abstractive Summarization](https://mlsanity.com/paper/2305.19783/) (2023)
- [BioLORD: Learning Ontological Representations from Definitions (for Biomedical Concepts and their Textual Descriptions)](https://mlsanity.com/paper/2210.11892/) (2022)
- [Design of Negative Sampling Strategies for Distantly Supervised Skill Extraction](https://mlsanity.com/paper/2209.05987/) (2022)
- [Large Language Models as Batteries-Included Zero-Shot ESCO Skills Matchers](https://mlsanity.com/paper/2307.03539/) (2023)
- [BioDEX: Large-Scale Biomedical Adverse Drug Event Extraction for Real-World Pharmacovigilance](https://mlsanity.com/paper/2305.13395/) (2023)

## Built on by (most cited first, 4 of 4)

- [Efficient Text Encoders for Labor Market Analysis](https://mlsanity.com/paper/2505.24640/) (2025)
- [A Multi-Stage Framework with Taxonomy-Guided Reasoning for Occupation Classification Using Large Language Models](https://mlsanity.com/paper/2503.12989/) (2025)
- [Dr. SoW: Density Ratio of Strong-over-weak LLMs for Reducing the Cost of Human Annotation in Preference Tuning](https://mlsanity.com/paper/2411.02481/) (2024)
- [Privacy Vulnerabilities of Attention Layers in Tabular Foundation Models and Protection of High-Risk Queries](https://mlsanity.com/paper/2606.26021/) (2026)

## Abstract (arXiv)

Multi-label classification problems with thousands of classes are hard to solve with in-context learning alone, as language models (LMs) might lack prior knowledge about the precise classes or how to assign them, and it is generally infeasible to demonstrate every class in a prompt. We propose a general program, textttInfer--Retrieve--Rank, that defines multi-step interactions between LMs and retrievers to efficiently tackle such problems. We implement this program using the textttDSPy programming model, which specifies in-context systems in a declarative manner, and use textttDSPy optimizers to tune it towards specific datasets by bootstrapping only tens of few-shot examples. Our primary extreme classification program, optimized separately for each task, attains state-of-the-art results across three benchmarks (HOUSE, TECH, TECHWOLF). We apply the same program to a benchmark with vastly different characteristics and attain competitive performance as well (BioDEX). Unlike prior work, our proposed solution requires no finetuning, is easily applicable to new tasks, alleviates prompt engineering, and requires only tens of labeled examples. Our code is public at https://github.com/KarelDO/xmc.dspy.