mlsanityData to 7 Oct 2026

Crowdsourcing in Computer Vision

arXiv 1611.02145 ↗First posted 7 Nov 2016cs.CV cs.HCcited by 31

Not on the map. Why?

Abstract (arXiv)

Computer vision systems require large amounts of manually annotated data to properly learn challenging visual concepts. Crowdsourcing platforms offer an inexpensive method to capture human knowledge and understanding, for a vast number of visual perception tasks. In this survey, we describe the types of annotations computer vision researchers have collected using crowdsourcing, and how they have ensured that this data is of high quality while annotation effort is minimized. We begin by discussing data collection on both classic (e.g., object recognition) and recent (e.g., visual story-telling) vision tasks. We then summarize key design decisions for creating effective data collection interfaces and workflows, and present strategies for intelligently selecting the most important data instances to annotate. Finally, we conclude with some thoughts on the future of crowdsourcing in computer vision.

Key facts. Cited by 31 papers in the map (data through 7 Oct 2026). First posted on arXiv 7 Nov 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 7; 2022: 8; 2023: 7; 2024: 3; 2025: 5; 2026: 1.

31 papers.

Sort:

By area:

  1. 2021NuCLS: A scalable crowdsourcing, deep learning approach and dataset for nucleus classification, localization and segmentationcited by 33
  2. 2023A comprehensive survey on deep active learning in medical image analysiscited by 32
  3. 2022A Survey on Deep Learning for Skin Lesion Segmentationcited by 22
  4. 2022From colouring-in to pointillism: revisiting semantic segmentation supervisioncited by 22
  5. 2024Conformalized Credal Set Predictorscited by 21
  6. 2023Streaming Active Learning with Deep Neural Networkscited by 20
  7. 2021Towards Good Practices for Efficiently Annotating Large-Scale Image Classification Datasetscited by 17
  8. 2024Decoupling the Class Label and the Target Concept in Machine Unlearningcited by 12
  9. 2022Knowledge Learning with Crowdsourcing: A Brief Review and Systematic Perspectivecited by 11
  10. 2022Towards Better User Studies in Computer Graphics and Visioncited by 11
  11. 2021Active Learning for Improved Semi-Supervised Semantic Segmentation in Satellite Imagescited by 9
  12. 2023Evaluating AI systems under uncertain ground truth: a case study in dermatologycited by 7
19 more
  1. 2022ALBench: A Framework for Evaluating Active Learning in Object Detectioncited by 6
  2. 2023Augmenting Chest X-ray Datasets with Non-Expert Annotationscited by 6
  3. 2021MEAL: Manifold Embedding-based Active Learningcited by 5
  4. 2021In Search of Ambiguity: A Three-Stage Workflow Design to Clarify Annotation Guidelines for Crowd Workerscited by 5
  5. 2022Improved Adaptive Algorithm for Scalable Active Learning with Weak Labelercited by 4
  6. 2023Combating Exacerbated Heterogeneity for Robust Models in Federated Learningcited by 4
  7. 2025Active Measurement: Efficient Estimation at Scalecited by 4
  8. 2021EventAnchor: Reducing Human Interactions in Event Annotation of Racket Sports Videoscited by 3
  9. 2025Optimal Conformal Prediction under Epistemic Uncertaintycited by 3
  10. 2023Multiview Representation Learning from Crowdsourced Triplet Comparisonscited by 2
  11. 2025A Generalizable Light Transport 3D Embedding for Global Illuminationcited by 2
  12. 2026Incentive-Tuning: Understanding and Designing Incentives for Empirical Human-AI Decision-Making Studiescited by 2
  13. 2021Semiotically-grounded distant viewing of diagrams: insights from two multimodal corporacited by 1
  14. 2022Resolving the Human Subjects Status of Machine Learning's Crowdworkerscited by 1
  15. 2023Manifold Path Guiding for Importance Sampling Specular Chainscited by 1
  16. 2022Fine-Grained Counting with Crowd-Sourced Supervisioncited by 0
  17. 2024Design and Evaluation of Camera-Centric Mobile Crowdsourcing Applicationscited by 0
  18. 2025From Label Error Detection to Correction: A Modular Framework and Benchmark for Object Detection Datasetscited by 0
  19. 2025Agile Deliberation: Concept Deliberation for Subjective Visual Classificationcited by 0