# Detecting adversaries in Crowdsourcing

arXiv: https://arxiv.org/abs/2110.04117 · HTML: https://mlsanity.com/paper/2110.04117/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 42 references that are in the map.
- First posted on arXiv 7 Oct 2021.

## Builds on (most cited first, 15 of 15)

- [Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation](https://mlsanity.com/paper/2010.16061/) (2020)
- [Spectral Methods meet EM: A Provably Optimal Algorithm for Crowdsourcing](https://mlsanity.com/paper/1406.3824/) (2014)
- [Oracle Based Active Set Algorithm for Scalable Elastic Net Subspace Clustering](https://mlsanity.com/paper/1605.02633/) (2016)
- [Crowdsourcing via Pairwise Co-occurrences: Identifiability and Algorithms](https://mlsanity.com/paper/1909.12325/) (2019)
- [Adversarial Crowdsourcing Through Robust Rank-One Matrix Completion](https://mlsanity.com/paper/2010.12181/) (2020)
- [Estimating the Accuracies of Multiple Classifiers Without Labeled Data](https://mlsanity.com/paper/1407.7644/) (2014)
- [Greedy Feature Selection for Subspace Clustering](https://mlsanity.com/paper/1303.4778/) (2013)
- [Sketched Subspace Clustering](https://mlsanity.com/paper/1707.07196/) (2017)
- [Gradient Descent for Sparse Rank-One Matrix Completion for Crowd-Sourced Aggregation of Sparsely Interacting Workers](https://mlsanity.com/paper/1904.11608/) (2019)
- [Greedy Subspace Clustering](https://mlsanity.com/paper/1410.8864/) (2014)
- [Certifiably Optimal Low Rank Factor Analysis](https://mlsanity.com/paper/1604.06837/) (2016)
- [Blind Multiclass Ensemble Classification](https://mlsanity.com/paper/1712.02903/) (2017)
- [Static and Dynamic Robust PCA and Matrix Completion: A Review](https://mlsanity.com/paper/1803.00651/) (2018)
- [Algebraic Clustering of Affine Subspaces](https://mlsanity.com/paper/1509.06729/) (2015)
- [Efficient Solvers for Sparse Subspace Clustering](https://mlsanity.com/paper/1804.06291/) (2018)

## Built on by (most cited first, 1 of 1)

- [Learning From Crowdsourced Noisy Labels: A Signal Processing Perspective](https://mlsanity.com/paper/2407.06902/) (2024)

## Abstract (arXiv)

Despite its successes in various machine learning and data science tasks, crowdsourcing can be susceptible to attacks from dedicated adversaries. This work investigates the effects of adversaries on crowdsourced classification, under the popular Dawid and Skene model. The adversaries are allowed to deviate arbitrarily from the considered crowdsourcing model, and may potentially cooperate. To address this scenario, we develop an approach that leverages the structure of second-order moments of annotator responses, to identify large numbers of adversaries, and mitigate their impact on the crowdsourcing task. The potential of the proposed approach is empirically demonstrated on synthetic and real crowdsourcing datasets.