mlsanityData to 14 Oct 2026

Knowledge Learning with Crowdsourcing: A Brief Review and Systematic Perspective

arXiv 2206.09315 ↗First posted 19 Jun 2022cs.LG cs.AI cs.HCcited by 11

Not on the map. Why?

Abstract (arXiv)

Big data have the characteristics of enormous volume, high velocity, diversity, value-sparsity, and uncertainty, which lead the knowledge learning from them full of challenges. With the emergence of crowdsourcing, versatile information can be obtained on-demand so that the wisdom of crowds is easily involved to facilitate the knowledge learning process. During the past thirteen years, researchers in the AI community made great efforts to remove the obstacles in the field of learning from crowds. This concentrated survey paper comprehensively reviews the technical progress in crowdsourcing learning from a systematic perspective that includes three dimensions of data, models, and learning processes. In addition to reviewing existing important work, the paper places a particular emphasis on providing some promising blueprints on each dimension as well as discussing the lessons learned from our past research work, which will light up the way for new researchers and encourage them to pursue new contributions.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Builds on 12 of its 116 references that are in the map. First posted on arXiv 19 Jun 2022.

Builds on

12 of 116 references.

References · 12

  1. 2019A Comprehensive Survey on Graph Neural Networkscited by 2,919
  2. 2016Data Programming: Creating Large Training Sets, Quicklycited by 246
  3. 2017Deep learning from crowdscited by 105
  4. 2014Spectral Methods meet EM: A Provably Optimal Algorithm for Crowdsourcingcited by 63
  5. 2016Crowdsourcing in Computer Visioncited by 31
  6. 2014PU Learning for Matrix Completioncited by 30
  7. 2019Max-MIG: an Information Theoretic Approach for Joint Learning from Crowdscited by 26
  8. 2020Nearly Optimal Regret for Stochastic Linear Bandits with Heavy-Tailed Payoffscited by 25
  9. 2019Graph convolutional networks for learning with few clean and many noisy labelscited by 8
  10. 2019Active Multi-Label Crowd Consensuscited by 6
  11. 2015Crowdsourcing Feature Discovery via Adaptively Chosen Comparisonscited by 5
  12. 2013Crowdsourcing for Bioinformaticscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 5; 2024: 2; 2025: 3; 2026: 1.

11 papers.

  1. 2025Harnessing Multiple Large Language Models: A Survey on LLM Ensemblecited by 86
  2. 2023Transferring Annotator- and Instance-dependent Transition Matrix for Learning from Crowdscited by 11
  3. 2023Coupled Confusion Correction: Learning from Crowds with Sparse Annotationscited by 9
  4. 2023Multi-annotator Deep Learning: A Probabilistic Framework for Classificationcited by 8
  5. 2024Learning with Confidence: Training Better Classifiers from Soft Labelscited by 6
  6. 2024SkillAggregation: Reference-free LLM-Dependent Aggregationcited by 6
  7. 2025CrowdAgent: Multi-Agent Managed Multi-Source Annotation Systemcited by 5
  8. 2023Neural-Hidden-CRF: A Robust Weakly-Supervised Sequence Labelercited by 3
  9. 2023Expertise-based Weighting for Regression Models with Noisy Labelscited by 0
  10. 2025Granular feedback merits sophisticated aggregationcited by 0
  11. 2026Unsupervised Ensemble Learning Through Deep Energy-based Modelscited by 0