# Active Learning by Feature Mixing

arXiv: https://arxiv.org/abs/2203.07034 · HTML: https://mlsanity.com/paper/2203.07034/

## Key facts

- Cited by 63 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 46 references that are in the map.
- First posted on arXiv 14 Mar 2022.

## Builds on (most cited first, 25 of 25)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [Convex optimization](https://mlsanity.com/paper/2106.01946/) (2021)
- [Decoupled Weight Decay Regularization](https://mlsanity.com/paper/1711.05101/) (2017)
- [Moment Matching for Multi-Source Domain Adaptation](https://mlsanity.com/paper/1812.01754/) (2018)
- [DeepFool: a simple and accurate method to fool deep neural networks](https://mlsanity.com/paper/1511.04599/) (2015)
- [Multiscale Vision Transformers](https://mlsanity.com/paper/2104.11227/) (2021)
- [Deep Bayesian Active Learning with Image Data](https://mlsanity.com/paper/1703.02910/) (2017)
- [Deep Batch Active Learning by Diverse, Uncertain Gradient Lower Bounds](https://mlsanity.com/paper/1906.03671/) (2019)
- [Domain Generalization with MixStyle](https://mlsanity.com/paper/2104.02008/) (2021)
- [Learning Loss for Active Learning](https://mlsanity.com/paper/1905.03677/) (2019)
- [Variational Adversarial Active Learning](https://mlsanity.com/paper/1904.00370/) (2019)
- [Interpolation Consistency Training for Semi-Supervised Learning](https://mlsanity.com/paper/1903.03825/) (2019)
- [Weakly-Supervised Disentanglement Without Compromises](https://mlsanity.com/paper/2002.02886/) (2020)
- [GENESIS: Generative Scene Inference and Sampling with Object-Centric Latent Representations](https://mlsanity.com/paper/1907.13052/) (2019)
- [Adversarial Active Learning for Deep Networks: a Margin Based Approach](https://mlsanity.com/paper/1802.09841/) (2018)
- [Contextual Diversity for Active Learning](https://mlsanity.com/paper/2008.05723/) (2020)
- [Task-Aware Variational Adversarial Active Learning](https://mlsanity.com/paper/2002.04709/) (2020)
- [Sequential Graph Convolutional Network for Active Learning](https://mlsanity.com/paper/2006.10219/) (2020)
- [VaB-AL: Incorporating Class Imbalance and Difficulty with Variational Bayes for Active Learning](https://mlsanity.com/paper/2003.11249/) (2020)

## Built on by (most cited first, 30 of 63)

- [Unsupervised Object Localization: Observing the Background to Discover Objects](https://mlsanity.com/paper/2212.07834/) (2022)
- [A Survey on Deep Active Learning: Recent Advances and New Frontiers](https://mlsanity.com/paper/2405.00334/) (2024)
- [AbdomenAtlas-8K: Annotating 8,000 CT Volumes for Multi-Organ Segmentation in Three Weeks](https://mlsanity.com/paper/2305.09666/) (2023)
- [Active Finetuning: Exploiting Annotation Budget in the Pretraining-Finetuning Paradigm](https://mlsanity.com/paper/2303.14382/) (2023)
- [Active Generalized Category Discovery](https://mlsanity.com/paper/2403.04272/) (2024)
- [Think Twice Before Selection: Federated Evidential Active Learning for Medical Image Analysis with Domain Shifts](https://mlsanity.com/paper/2312.02567/) (2023)
- [Exploring Active 3D Object Detection from a Generalization Perspective](https://mlsanity.com/paper/2301.09249/) (2023)
- [Active Prompt Learning in Vision Language Models](https://mlsanity.com/paper/2311.11178/) (2023)
- [Knowledge-Aware Federated Active Learning with Non-IID Data](https://mlsanity.com/paper/2211.13579/) (2022)
- [ActiveAD: Planning-Oriented Active Learning for End-to-End Autonomous Driving](https://mlsanity.com/paper/2403.02877/) (2024)
- [Generalized Coverage for More Robust Low-Budget Active Learning](https://mlsanity.com/paper/2407.12212/) (2024)
- [Active Learning for Vision-Language Models](https://mlsanity.com/paper/2410.22187/) (2024)
- [Uncertainty Herding: One Active Learning Method for All Label Budgets](https://mlsanity.com/paper/2412.20644/) (2024)
- [Re-thinking Federated Active Learning based on Inter-class Diversity](https://mlsanity.com/paper/2303.12317/) (2023)
- [Revisiting Active Learning in the Era of Vision Foundation Models](https://mlsanity.com/paper/2401.14555/) (2024)
- [KECOR: Kernel Coding Rate Maximization for Active 3D Object Detection](https://mlsanity.com/paper/2307.07942/) (2023)
- [Divide and Adapt: Active Domain Adaptation via Customized Learning](https://mlsanity.com/paper/2307.11618/) (2023)
- [TiDAL: Learning Training Dynamics for Active Learning](https://mlsanity.com/paper/2210.06788/) (2022)
- [Towards Balanced Active Learning for Multimodal Classification](https://mlsanity.com/paper/2306.08306/) (2023)
- [A Survey of Dataset Refinement for Problems in Computer Vision Datasets](https://mlsanity.com/paper/2210.11717/) (2022)
- [Active Learning via Classifier Impact and Greedy Selection for Interactive Image Retrieval](https://mlsanity.com/paper/2412.02310/) (2024)
- [UP-DP: Unsupervised Prompt Learning for Data Pre-Selection with Vision-Language Models](https://mlsanity.com/paper/2307.11227/) (2023)
- [Overcoming Overconfidence for Active Learning](https://mlsanity.com/paper/2308.10571/) (2023)
- [Learning to Rank for Active Learning via Multi-Task Bilevel Optimization](https://mlsanity.com/paper/2310.17044/) (2023)
- [ActiveDC: Distribution Calibration for Active Finetuning](https://mlsanity.com/paper/2311.07634/) (2023)
- [Downstream-Pretext Domain Knowledge Traceback for Active Learning](https://mlsanity.com/paper/2407.14720/) (2024)
- [FAST: Federated Active Learning with Foundation Models for Communication-efficient Sampling and Training](https://mlsanity.com/paper/2504.03783/) (2025)
- [Active Semi-Supervised Learning by Exploring Per-Sample Uncertainty and Consistency](https://mlsanity.com/paper/2303.08978/) (2023)
- [Training Ensembles with Inliers and Outliers for Semi-supervised Active Learning](https://mlsanity.com/paper/2307.03741/) (2023)
- [A Pre-trained Data Deduplication Model based on Active Learning](https://mlsanity.com/paper/2308.00721/) (2023)

## Abstract (arXiv)

The promise of active learning (AL) is to reduce labelling costs by selecting the most valuable examples to annotate from a pool of unlabelled data. Identifying these examples is especially challenging with high-dimensional data (e.g. images, videos) and in low-data regimes. In this paper, we propose a novel method for batch AL called ALFA-Mix. We identify unlabelled instances with sufficiently-distinct features by seeking inconsistencies in predictions resulting from interventions on their representations. We construct interpolations between representations of labelled and unlabelled instances then examine the predicted labels. We show that inconsistencies in these predictions help discovering features that the model is unable to recognise in the unlabelled instances. We derive an efficient implementation based on a closed-form solution to the optimal interpolation causing changes in predictions. Our method outperforms all recent AL approaches in 30 different settings on 12 benchmarks of images, videos, and non-visual data. The improvements are especially significant in low-data regimes and on self-trained vision transformers, where ALFA-Mix outperforms the state-of-the-art in 59% and 43% of the experiments respectively.