mlsanityData to 13 Oct 2026

Agile Modeling: From Concept to Classifier in Minutes

arXiv 2302.12948 ↗First posted 25 Feb 2023cs.LG cs.AI cs.CVcited by 8

Not on the map. Why?

Abstract (arXiv)

The application of computer vision to nuanced subjective use cases is growing. While crowdsourcing has served the vision community well for most objective tasks (such as labeling a "zebra"), it now falters on tasks where there is substantial subjectivity in the concept (such as identifying "gourmet tuna"). However, empowering any user to develop a classifier for their concept is technically difficult: users are neither machine learning experts, nor have the patience to label thousands of examples. In reaction, we introduce the problem of Agile Modeling: the process of turning any subjective visual concept into a computer vision model through a real-time user-in-the-loop interactions. We instantiate an Agile Modeling prototype for image classification and show through a user study (N=14) that users can create classifiers with minimal effort under 30 minutes. We compare this user driven process with the traditional crowdsourcing paradigm and find that the crowd's notion often differs from that of the user's, especially as the concepts become more subjective. Finally, we scale our experiments with simulations of users training classifiers for ImageNet21k categories to further demonstrate the efficacy.

Key facts. Cited by 8 papers in the map (data through 13 Oct 2026). Builds on 32 of its 81 references that are in the map. First posted on arXiv 25 Feb 2023.

Builds on

32 of 81 references.

References · 32

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,093
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,588
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,550
  4. 2022Training language models to follow instructions with human feedbackcited by 15,856
  5. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,476
  6. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,870
  7. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,141
  8. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,855
  9. 2017Prototypical Networks for Few-shot Learningcited by 3,011
  10. 2016Matching Networks for One Shot Learningcited by 2,251
  11. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,240
  12. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,548
20 more
  1. 2020Big Self-Supervised Models are Strong Semi-Supervised Learnerscited by 1,378
  2. 2019Deep Batch Active Learning by Diverse, Uncertain Gradient Lower Boundscited by 606
  3. 2020A Survey of Deep Active Learningcited by 550
  4. 2020Rethinking Few-Shot Image Classification: a Good Embedding Is All You Need?cited by 436
  5. 2017Snorkel: Rapid Training Data Creation with Weak Supervisioncited by 371
  6. 2019Selection via Proxy: Efficient Data Selection for Deep Learningcited by 339
  7. 2019Meta-Dataset: A Dataset of Datasets for Learning to Learn from Few Examplescited by 314
  8. 2022What does a platypus look like? Generating customized prompts for zero-shot image classificationcited by 264
  9. 2021Batch Active Learning at Scalecited by 127
  10. 2017Deep Active Learning for Named Entity Recognitioncited by 108
  11. 2022"This is my unicorn, Fluffy": Personalizing frozen vision-language representationscited by 95
  12. 2019Bayesian Batch Active Learning as Sparse Subset Approximationcited by 68
  13. 2021Mind Your Outliers! Investigating the Negative Impact of Outliers on Active Learning for Visual Question Answeringcited by 62
  14. 2015Quantization based Fast Inner Product Searchcited by 29
  15. 2020Similarity Search for Efficient Active Learning and Search of Rare Conceptscited by 24
  16. 2017Inferring Generative Model Structure with Static Analysiscited by 18
  17. 2023EQUI-VOCAL: Synthesizing Queries for Compositional Video Events from Limited User Interactions [Technical Report]cited by 10
  18. 2020Background Splitting: Finding Rare Classes in a Sea of Backgroundcited by 5
  19. 2021Personalizing Pre-trained Modelscited by 5
  20. 2021Sketch-based Creativity Support Tools using Deep Learningcited by 3

Built on by

Cited by year

Citing papers per year: 2023: 4; 2024: 2; 2025: 2.

8 papers.

  1. 2023Visual Program Distillation: Distilling Tools and Programmatic Reasoning into Vision-Language Modelscited by 66
  2. 2024m&m's: A Benchmark to Evaluate Tool-Use for multi-step multi-modal Taskscited by 38
  3. 2023Designing LLM Chains by Adapting Techniques from Crowdsourcing Workflowscited by 17
  4. 2023Semantic and Expressive Variation in Image Captions Across Languagescited by 8
  5. 2024Leveraging tropical reef, bird and unrelated sounds for superior transfer learning in marine bioacousticscited by 7
  6. 2023From Categories to Classifiers: Name-Only Continual Learning by Exploring the Webcited by 5
  7. 2025A Zero-Shot Learning Approach for Ephemeral Gully Detection from Remote Sensing using Vision Language Modelscited by 1
  8. 2025Agile Deliberation: Concept Deliberation for Subjective Visual Classificationcited by 0