# Crowdsourcing Learning as Domain Adaptation: A Case Study on Named Entity Recognition

arXiv: https://arxiv.org/abs/2105.14980 · HTML: https://mlsanity.com/paper/2105.14980/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 64 references that are in the map.
- First posted on arXiv 31 May 2021.

## Builds on (most cited first, 15 of 15)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Parameter-Efficient Transfer Learning for NLP](https://mlsanity.com/paper/1902.00751/) (2019)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [Neural Architectures for Named Entity Recognition](https://mlsanity.com/paper/1603.01360/) (2016)
- [Neural Unsupervised Domain Adaptation in NLP---A Survey](https://mlsanity.com/paper/2006.00632/) (2020)
- [Domain Adaptation for Visual Applications: A Comprehensive Survey](https://mlsanity.com/paper/1702.05374/) (2017)
- [A Survey on Recent Advances in Named Entity Recognition from Deep Learning models](https://mlsanity.com/paper/1910.11470/) (2019)
- [Deep learning from crowds](https://mlsanity.com/paper/1709.01779/) (2017)
- [A Corpus with Multi-Level Annotations of Patients, Interventions and Outcomes to Support Language Processing for Medical Literature](https://mlsanity.com/paper/1806.04185/) (2018)
- [UDapter: Language Adaptation for Truly Universal Dependency Parsing](https://mlsanity.com/paper/2004.14327/) (2020)
- [A Survey of Domain Adaptation for Neural Machine Translation](https://mlsanity.com/paper/1806.00258/) (2018)
- [Contextual Parameter Generation for Universal Neural Machine Translation](https://mlsanity.com/paper/1808.08493/) (2018)
- [Fine-Grained Argument Unit Recognition and Classification](https://mlsanity.com/paper/1904.09688/) (2019)
- [Robust Named Entity Recognition with Truecasing Pretraining](https://mlsanity.com/paper/1912.07095/) (2019)
- [Adversarial Learning for Chinese NER from Crowd Annotations](https://mlsanity.com/paper/1801.05147/) (2018)

## Built on by (most cited first, 13 of 13)

- [PandaLM: An Automatic Evaluation Benchmark for LLM Instruction Tuning Optimization](https://mlsanity.com/paper/2306.05087/) (2023)
- [Annotation Error Detection: Analyzing the Past and Present for a More Coherent Future](https://mlsanity.com/paper/2206.02280/) (2022)
- [Character-level White-Box Adversarial Attacks against Transformers via Attachable Subwords Substitution](https://mlsanity.com/paper/2210.17004/) (2022)
- [Unsupervised Entity Alignment for Temporal Knowledge Graphs](https://mlsanity.com/paper/2302.00796/) (2023)
- [Searching for Optimal Subword Tokenization in Cross-domain NER](https://mlsanity.com/paper/2206.03352/) (2022)
- [A Frustratingly Easy Plug-and-Play Detection-and-Reasoning Module for Chinese Spelling Check](https://mlsanity.com/paper/2310.09119/) (2023)
- [A Data Fusion Framework for Multi-Domain Morality Learning](https://mlsanity.com/paper/2304.02144/) (2023)
- [Domain-Specific NER via Retrieving Correlated Samples](https://mlsanity.com/paper/2208.12995/) (2022)
- [Identifying Chinese Opinion Expressions with Extremely-Noisy Crowdsourcing Annotations](https://mlsanity.com/paper/2204.10714/) (2022)
- [Query-based Instance Discrimination Network for Relational Triple Extraction](https://mlsanity.com/paper/2211.01797/) (2022)
- [Neural-Hidden-CRF: A Robust Weakly-Supervised Sequence Labeler](https://mlsanity.com/paper/2309.05086/) (2023)
- [Scene Graph Modification as Incremental Structure Expanding](https://mlsanity.com/paper/2209.09093/) (2022)
- [Universal Multi-modal Entity Alignment via Iteratively Fusing Modality Similarity Paths](https://mlsanity.com/paper/2310.05364/) (2023)

## Abstract (arXiv)

Crowdsourcing is regarded as one prospective solution for effective supervised learning, aiming to build large-scale annotated training data by crowd workers. Previous studies focus on reducing the influences from the noises of the crowdsourced annotations for supervised models. We take a different point in this work, regarding all crowdsourced annotations as gold-standard with respect to the individual annotators. In this way, we find that crowdsourcing could be highly similar to domain adaptation, and then the recent advances of cross-domain methods can be almost directly applied to crowdsourcing. Here we take named entity recognition (NER) as a study case, suggesting an annotator-aware representation learning model that inspired by the domain adaptation methods which attempt to capture effective domain-aware features. We investigate both unsupervised and supervised crowdsourcing learning, assuming that no or only small-scale expert annotations are available. Experimental results on a benchmark crowdsourced NER dataset show that our method is highly effective, leading to a new state-of-the-art performance. In addition, under the supervised setting, we can achieve impressive performance gains with only a very small scale of expert annotations.