mlsanityData to 14 Oct 2026

ADROIT: A Self-Supervised Framework for Learning Robust Representations for Active Learning

arXiv 2503.07506 ↗First posted 10 Mar 2025cs.LG cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Active learning aims to select optimal samples for labeling, minimizing annotation costs. This paper introduces a unified representation learning framework tailored for active learning with task awareness. It integrates diverse sources, comprising reconstruction, adversarial, self-supervised, knowledge-distillation, and classification losses into a unified VAE-based ADROIT approach. The proposed approach comprises three key components - a unified representation generator (VAE), a state discriminator, and a (proxy) task-learner or classifier. ADROIT learns a latent code using both labeled and unlabeled data, incorporating task-awareness by leveraging labeled data with the proxy classifier. Unlike previous approaches, the proxy classifier additionally employs a self-supervised loss on unlabeled data and utilizes knowledge distillation to align with the target task-learner. The state discriminator distinguishes between labeled and unlabeled data, facilitating the selection of informative unlabeled samples. The dynamic interaction between VAE and the state discriminator creates a competitive environment, with the VAE attempting to deceive the discriminator, while the state discriminator learns to differentiate between labeled and unlabeled inputs. Extensive evaluations on diverse datasets and ablation analysis affirm the effectiveness of the proposed model.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 59 of its 83 references that are in the map. First posted on arXiv 10 Mar 2025.

Builds on

59 of 83 references.

References · 59

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  4. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  5. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  6. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,527
  7. 2020Exploring Simple Siamese Representation Learningcited by 2,866
  8. 2014Conditional Generative Adversarial Netscited by 2,162
  9. 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,054
  10. 2014Unsupervised Domain Adaptation by Backpropagationcited by 1,917
  11. 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,591
  12. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,387
38 more
  1. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,382
  2. 2021Variational Diffusion Modelscited by 1,153
  3. 2015Unsupervised Visual Representation Learning by Context Predictioncited by 1,066
  4. 2017Deep Bayesian Active Learning with Image Datacited by 819
  5. 2021VideoGPT: Video Generation using VQ-VAE and Transformerscited by 658
  6. 2016Improving Variational Inference with Inverse Autoregressive Flowcited by 640
  7. 2020NVAE: A Deep Hierarchical Variational Autoencodercited by 637
  8. 2020A Survey of Deep Active Learningcited by 554
  9. 2014Semi-Supervised Learning with Deep Generative Modelscited by 496
  10. 2019BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learningcited by 432
  11. 2016Tutorial on Variational Autoencoderscited by 409
  12. 2017Wasserstein Auto-Encoderscited by 373
  13. 2016Adversarial Feature Learningcited by 350
  14. 2022Understanding Diffusion Models: A Unified Perspectivecited by 340
  15. 2019Learning Loss for Active Learningcited by 336
  16. 2019Variational Adversarial Active Learningcited by 284
  17. 2017Data Augmentation Generative Adversarial Networkscited by 278
  18. 2023A Cookbook of Self-Supervised Learningcited by 250
  19. 2018Unsupervised Anomaly Detection via Variational Auto-Encoder for Seasonal KPIs in Web Applicationscited by 224
  20. 2022Diffusion-GAN: Training GANs with Diffusioncited by 205
  21. 2016Variational Lossy Autoencodercited by 176
  22. 2021Efficient Self-supervised Vision Transformers for Representation Learningcited by 149
  23. 2020Contextual Diversity for Active Learningcited by 139
  24. 2019Consistency-based Semi-supervised Active Learning: Towards Minimizing Labeling Costcited by 114
  25. 2018Fast Decoding in Sequence Models using Discrete Latent Variablescited by 104
  26. 2016Semi-Supervised Learning with Generative Adversarial Networkscited by 103
  27. 2019Deep Active Learning: Unified and Principled Method for Query and Trainingcited by 93
  28. 2022A Comparative Survey of Deep Active Learningcited by 89
  29. 2020Task-Aware Variational Adversarial Active Learningcited by 84
  30. 2017Generative Adversarial Active Learningcited by 70
  31. 2020State-Relabeling Adversarial Active Learningcited by 60
  32. 2021Semi-Supervised Active Learning with Temporal Output Discrepancycited by 50
  33. 2016Stacked Generative Adversarial Networkscited by 44
  34. 2018Conditional Image-to-Image Translationcited by 30
  35. 2017Variational Attention for Sequence-to-Sequence Modelscited by 29
  36. 2022A simple, efficient and scalable contrastive masked autoencoder for learning visual representationscited by 25
  37. 2018DA-GAN: Instance-level Image Translation by Deep Attention Generative Adversarial Networks (with Supplementary Materials)cited by 21
  38. 2017Semantically Decomposing the Latent Spaces of Generative Adversarial Networkscited by 19

50 most cited of 59 All 59 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025PCoreSet: Effective Active Learning through Knowledge Distillation from Vision-Language Models
  2. 2025Active Learning via Vision-Language Model Adaptation with Open Data
  3. 2023Sample-efficient Adversarial Imitation Learning
  4. 2024Revisiting Active Learning in the Era of Vision Foundation Models
  5. 2025Boosting Active Learning with Knowledge Transfer
  6. 2026Activation-Based Active Learning for In-Context Learning: Challenges and Insights
  7. 2022Robust Active Distillation
  8. 2022ARDIR: Improving Robustness using Knowledge Distillation of Internal Representation