mlsanityData to 7 Oct 2026

Bridging Diversity and Uncertainty in Active learning with Self-Supervised Pre-Training

arXiv 2403.03728 ↗First posted 6 Mar 2024cs.LG cs.AI cs.CVcited by 6

Not on the map. Why?

Abstract (arXiv)

This study addresses the integration of diversity-based and uncertainty-based sampling strategies in active learning, particularly within the context of self-supervised pre-trained models. We introduce a straightforward heuristic called TCM that mitigates the cold start problem while maintaining strong performance across various data levels. By initially applying TypiClust for diversity sampling and subsequently transitioning to uncertainty sampling with Margin, our approach effectively combines the strengths of both strategies. Our experiments demonstrate that TCM consistently outperforms existing methods across various datasets in both low and high data regimes.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 11 of its 16 references that are in the map. First posted on arXiv 6 Mar 2024.

Builds on

11 of 16 references.

References · 11

  1. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  2. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  3. 2019Learning Imbalanced Datasets with Label-Distribution-Aware Margin Losscited by 987
  4. 2017Deep Bayesian Active Learning with Image Datacited by 812
  5. 2019Deep Batch Active Learning by Diverse, Uncertain Gradient Lower Boundscited by 602
  6. 2017Skin Lesion Analysis Toward Melanoma Detection: A Challenge at the 2017 International Symposium on Biomedical Imaging (ISBI), Hosted by the International Skin Imaging Collaboration (ISIC)cited by 549
  7. 2019BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learningcited by 428
  8. 2022Active Learning on a Budget: Opposite Strategies Suit High and Low Budgetscited by 131
  9. 2022Active Learning Through a Covering Lenscited by 62
  10. 2019Parting with Illusions about Deep Active Learningcited by 38
  11. 2023How to Select Which Active Learning Strategy is Best Suited for Your Specific Problem and Budgetcited by 16

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 4; 2026: 1.

6 papers.

  1. 2024Uncertainty Herding: One Active Learning Method for All Label Budgetscited by 12
  2. 2025Active Learning for Direct Preference Optimizationcited by 9
  3. 2025Sharpe Ratio-Guided Active Learning for Preference Optimization in RLHFcited by 5
  4. 2025Cleaning the Pool: Progressive Filtering of Unlabeled Pools in Deep Active Learningcited by 2
  5. 2025MedCAL-Bench: A Comprehensive Benchmark on Cold-Start Active Learning with Foundation Models for Medical Image Analysiscited by 1
  6. 2026Group-invariant Coresets for Data-efficient Active Learningcited by 0