# STENCIL: Submodular Mutual Information Based Weak Supervision for Cold-Start Active Learning

arXiv: https://arxiv.org/abs/2402.13468 · HTML: https://mlsanity.com/paper/2402.13468/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 23 of its 39 references that are in the map.
- First posted on arXiv 21 Feb 2024.

## Builds on (most cited first, 23 of 23)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Instruction Tuning for Large Language Models: A Survey](https://mlsanity.com/paper/2308.10792/) (2023)
- [Deep Batch Active Learning by Diverse, Uncertain Gradient Lower Bounds](https://mlsanity.com/paper/1906.03671/) (2019)
- [A Survey of Deep Active Learning](https://mlsanity.com/paper/2009.00236/) (2020)
- [Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Survey](https://mlsanity.com/paper/2111.01243/) (2021)
- [Active Learning on a Budget: Opposite Strategies Suit High and Low Budgets](https://mlsanity.com/paper/2202.02794/) (2022)
- [Cold-start Active Learning through Self-supervised Language Modeling](https://mlsanity.com/paper/2010.09535/) (2020)
- [Lazier Than Lazy Greedy](https://mlsanity.com/paper/1409.7938/) (2014)
- [Submodular Combinatorial Information Measures with Applications in Machine Learning](https://mlsanity.com/paper/2006.15412/) (2020)
- [SIMILAR: Submodular Information Measures Based Active Learning In Realistic Scenarios](https://mlsanity.com/paper/2107.00717/) (2021)
- [PRISM: A Rich Class of Parameterized Submodular Information Measures for Guided Subset Selection](https://mlsanity.com/paper/2103.00128/) (2021)
- [Effective Evaluation of Deep Active Learning on Image Classification Tasks](https://mlsanity.com/paper/2106.15324/) (2021)
- [TALISMAN: Targeted Active Learning for Object Detection with Rare Classes and Slices using Submodular Mutual Information](https://mlsanity.com/paper/2112.00166/) (2021)
- [Semi-Supervised Data Programming with Subset Selection](https://mlsanity.com/paper/2008.09887/) (2020)
- [Merging Weak and Active Supervision for Semantic Parsing](https://mlsanity.com/paper/1911.12986/) (2019)
- [Learning to Robustly Aggregate Labeling Functions for Semi-supervised Data Programming](https://mlsanity.com/paper/2109.11410/) (2021)
- [A Memoization Framework for Scaling Submodular Optimization to Large Scale Problems](https://mlsanity.com/paper/1902.10176/) (2019)
- [Submodlib: A Submodular Optimization Library](https://mlsanity.com/paper/2202.10680/) (2022)
- [Active Data Discovery: Mining Unknown Data using Submodular Information Measures](https://mlsanity.com/paper/2206.08566/) (2022)
- [Active learning for imbalanced data under cold start](https://mlsanity.com/paper/2107.07724/) (2021)
- [Beyond Active Learning: Leveraging the Full Potential of Human Interaction via Auto-Labeling, Human Correction, and Human Verification](https://mlsanity.com/paper/2306.01277/) (2023)
- [Cold Start Active Learning Strategies in the Context of Imbalanced Classification](https://mlsanity.com/paper/2201.10227/) (2022)

## Abstract (arXiv)

As supervised fine-tuning of pre-trained models within NLP applications increases in popularity, larger corpora of annotated data are required, especially with increasing parameter counts in large language models. Active learning, which attempts to mine and annotate unlabeled instances to improve model performance maximally fast, is a common choice for reducing the annotation cost; however, most methods typically ignore class imbalance and either assume access to initial annotated data or require multiple rounds of active learning selection before improving rare classes. We present STENCIL, which utilizes a set of text exemplars and the recently proposed submodular mutual information to select a set of weakly labeled rare-class instances that are then strongly labeled by an annotator. We show that STENCIL improves overall accuracy by 10%-18% and rare-class F-1 score by 17%-40% on multiple text classification datasets over common active learning methods within the class-imbalanced cold-start setting.