# SIMILAR: Submodular Information Measures Based Active Learning In Realistic Scenarios

arXiv: https://arxiv.org/abs/2107.00717 · HTML: https://mlsanity.com/paper/2107.00717/

## Key facts

- Cited by 84 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 47 references that are in the map.
- First posted on arXiv 1 Jul 2021.

## Builds on (most cited first, 22 of 22)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Books](https://mlsanity.com/paper/1506.06724/) (2015)
- [Deep Batch Active Learning by Diverse, Uncertain Gradient Lower Bounds](https://mlsanity.com/paper/1906.03671/) (2019)
- [BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learning](https://mlsanity.com/paper/1906.08158/) (2019)
- [Adversarial Autoencoders](https://mlsanity.com/paper/1511.05644/) (2015)
- [YouTube-8M: A Large-Scale Video Classification Benchmark](https://mlsanity.com/paper/1609.08675/) (2016)
- [One Billion Word Benchmark for Measuring Progress in Statistical Language Modeling](https://mlsanity.com/paper/1312.3005/) (2013)
- [Variational Adversarial Active Learning](https://mlsanity.com/paper/1904.00370/) (2019)
- [GLISTER: Generalization based Data Subset Selection for Efficient and Robust Learning](https://mlsanity.com/paper/2012.10630/) (2020)
- [Lazier Than Lazy Greedy](https://mlsanity.com/paper/1409.7938/) (2014)
- [Submodular Combinatorial Information Measures with Applications in Machine Learning](https://mlsanity.com/paper/2006.15412/) (2020)
- [Learning From Less Data: A Unified Data Subset Selection and Active Learning Framework for Computer Vision](https://mlsanity.com/paper/1901.01151/) (2019)
- [Deep Active Learning for Biased Datasets via Fisher Kernel Self-Supervision](https://mlsanity.com/paper/2003.00393/) (2020)
- [Submodular Functions: from Discrete to Continous Domains](https://mlsanity.com/paper/1511.00394/) (2015)
- [Submodularity in Action: From Machine Learning to Signal Processing Applications](https://mlsanity.com/paper/2006.09905/) (2020)
- [A Memoization Framework for Scaling Submodular Optimization to Large Scale Problems](https://mlsanity.com/paper/1902.10176/) (2019)

## Built on by (most cited first, 30 of 84)

- [DeepCore: A Comprehensive Library for Coreset Selection in Deep Learning](https://mlsanity.com/paper/2204.08499/) (2022)
- [A Comparative Survey of Deep Active Learning](https://mlsanity.com/paper/2203.13450/) (2022)
- [A Survey of Multimodal Large Language Model from A Data-centric Perspective](https://mlsanity.com/paper/2405.16640/) (2024)
- [Data Pruning via Moving-one-Sample-out](https://mlsanity.com/paper/2310.14664/) (2023)
- [FisherRF: Active View Selection and Uncertainty Quantification for Radiance Fields using Fisher Information](https://mlsanity.com/paper/2311.17874/) (2023)
- [A Survey on Deep Active Learning: Recent Advances and New Frontiers](https://mlsanity.com/paper/2405.00334/) (2024)
- [A comprehensive survey on deep active learning in medical image analysis](https://mlsanity.com/paper/2310.14230/) (2023)
- [GALAXY: Graph-based Active Learning at the Extreme](https://mlsanity.com/paper/2202.01402/) (2022)
- [Unifying Approaches in Active Learning and Active Sampling via Fisher Information and Information-Theoretic Quantities](https://mlsanity.com/paper/2208.00549/) (2022)
- [A Coreset Selection of Coreset Selection Literature: Introduction and Recent Advances](https://mlsanity.com/paper/2505.17799/) (2025)
- [TALISMAN: Targeted Active Learning for Object Detection with Rare Classes and Slices using Submodular Mutual Information](https://mlsanity.com/paper/2112.00166/) (2021)
- [Meta-Query-Net: Resolving Purity-Informativeness Dilemma in Open-set Active Learning](https://mlsanity.com/paper/2210.07805/) (2022)
- [Entropic Open-set Active Learning](https://mlsanity.com/paper/2312.14126/) (2023)
- [Algorithm Selection for Deep Active Learning with Imbalanced Datasets](https://mlsanity.com/paper/2302.07317/) (2023)
- [SMART: Submodular Data Mixture Strategy for Instruction Tuning](https://mlsanity.com/paper/2403.08370/) (2024)
- [Fast Rates in Pool-Based Batch Active Learning](https://mlsanity.com/paper/2202.05448/) (2022)
- [Data Pruning by Information Maximization](https://mlsanity.com/paper/2506.01701/) (2025)
- [Active Learning with Neural Networks: Insights from Nonparametric Statistics](https://mlsanity.com/paper/2210.08367/) (2022)
- [Black-Box Batch Active Learning for Regression](https://mlsanity.com/paper/2302.08981/) (2023)
- [Data pruning and neural scaling laws: fundamental limitations of score-based algorithms](https://mlsanity.com/paper/2302.06960/) (2023)
- [Learning Neural Set Functions Under the Optimal Subset Oracle](https://mlsanity.com/paper/2203.01693/) (2022)
- [Gaussian Switch Sampling: A Second Order Approach to Active Learning](https://mlsanity.com/paper/2302.12018/) (2023)
- [CLINICAL: Targeted Active Learning for Imbalanced Medical Image Classification](https://mlsanity.com/paper/2210.01520/) (2022)
- [A Weighted K-Center Algorithm for Data Subset Selection](https://mlsanity.com/paper/2312.10602/) (2023)
- [Beyond Efficiency: Molecular Data Pruning for Enhanced Generalization](https://mlsanity.com/paper/2409.01081/) (2024)
- [Active Fine-Tuning of Multi-Task Policies](https://mlsanity.com/paper/2410.05026/) (2024)
- [Forgetful Active Learning with Switch Events: Efficient Sampling for Out-of-Distribution Data](https://mlsanity.com/paper/2301.05106/) (2023)
- [Improved Algorithm for Deep Active Learning under Imbalance via Optimal Separation](https://mlsanity.com/paper/2312.09196/) (2023)
- [Deep Active Learning for Computer Vision: Past and Future](https://mlsanity.com/paper/2211.14819/) (2022)
- [A Unified Approach Towards Active Learning and Out-of-Distribution Detection](https://mlsanity.com/paper/2405.11337/) (2024)

## Abstract (arXiv)

Active learning has proven to be useful for minimizing labeling costs by selecting the most informative samples. However, existing active learning methods do not work well in realistic scenarios such as imbalance or rare classes, out-of-distribution data in the unlabeled set, and redundancy. In this work, we propose SIMILAR (Submodular Information Measures based actIve LeARning), a unified active learning framework using recently proposed submodular information measures (SIM) as acquisition functions. We argue that SIMILAR not only works in standard active learning, but also easily extends to the realistic settings considered above and acts as a one-stop solution for active learning that is scalable to large real-world datasets. Empirically, we show that SIMILAR significantly outperforms existing active learning algorithms by as much as ~5% - 18% in the case of rare classes and ~5% - 10% in the case of out-of-distribution data on several image classification tasks like CIFAR-10, MNIST, and ImageNet. SIMILAR is available as a part of the DISTIL toolkit: "https://github.com/decile-team/distil".