# Target-Oriented Fine-tuning for Zero-Resource Named Entity Recognition

arXiv: https://arxiv.org/abs/2107.10523 · HTML: https://mlsanity.com/paper/2107.10523/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 38 references that are in the map.
- First posted on arXiv 22 Jul 2021.

## Builds on (most cited first, 24 of 24)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [Don't Stop Pretraining: Adapt Language Models to Domains and Tasks](https://mlsanity.com/paper/2004.10964/) (2020)
- [SpanBERT: Improving Pre-training by Representing and Predicting Spans](https://mlsanity.com/paper/1907.10529/) (2019)
- [On the Cross-lingual Transferability of Monolingual Representations](https://mlsanity.com/paper/1910.11856/) (2019)
- [Bidirectional LSTM-CRF Models for Sequence Tagging](https://mlsanity.com/paper/1508.01991/) (2015)
- [Word Translation Without Parallel Data](https://mlsanity.com/paper/1710.04087/) (2017)
- [MAD-X: An Adapter-Based Framework for Multi-Task Cross-Lingual Transfer](https://mlsanity.com/paper/2005.00052/) (2020)
- [End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF](https://mlsanity.com/paper/1603.01354/) (2016)
- [MLQA: Evaluating Cross-lingual Extractive Question Answering](https://mlsanity.com/paper/1910.07475/) (2019)
- [NewsQA: A Machine Comprehension Dataset](https://mlsanity.com/paper/1611.09830/) (2016)
- [A Unified MRC Framework for Named Entity Recognition](https://mlsanity.com/paper/1910.11476/) (2019)
- [Three Mechanisms of Weight Decay Regularization](https://mlsanity.com/paper/1810.12281/) (2018)
- [Coreference Resolution as Query-based Span Prediction](https://mlsanity.com/paper/1911.01746/) (2019)
- [TWEETQA: A Social Media Focused Question Answering Dataset](https://mlsanity.com/paper/1907.06292/) (2019)
- [English Intermediate-Task Training Improves Zero-Shot Cross-Lingual Transfer Too](https://mlsanity.com/paper/2005.13013/) (2020)
- [Single-/Multi-Source Cross-Lingual NER via Teacher-Student Learning on Unlabeled Data in Target Language](https://mlsanity.com/paper/2004.12440/) (2020)
- [PERL: Pivot-based Domain Adaptation for Pre-trained Deep Contextualized Embedding Models](https://mlsanity.com/paper/2006.09075/) (2020)
- [UniTrans: Unifying Model Transfer and Data Transfer for Cross-Lingual Named Entity Recognition with Unlabeled Data](https://mlsanity.com/paper/2007.07683/) (2020)
- [Orthogonal Language and Task Adapters in Zero-Shot Cross-Lingual Transfer](https://mlsanity.com/paper/2012.06460/) (2020)
- [GCDT: A Global Context Enhanced Deep Transition Architecture for Sequence Labeling](https://mlsanity.com/paper/1906.02437/) (2019)
- [Zero-Resource Cross-Lingual Named Entity Recognition](https://mlsanity.com/paper/1911.09812/) (2019)
- [Zero-Resource Cross-Domain Named Entity Recognition](https://mlsanity.com/paper/2002.05923/) (2020)
- [Towards Lingua Franca Named Entity Recognition with BERT](https://mlsanity.com/paper/1912.01389/) (2019)

## Built on by (most cited first, 5 of 5)

- [ConNER: Consistency Training for Cross-lingual Named Entity Recognition](https://mlsanity.com/paper/2211.09394/) (2022)
- [WIDER & CLOSER: Mixture of Short-channel Distillers for Zero-shot Cross-lingual Named Entity Recognition](https://mlsanity.com/paper/2212.03506/) (2022)
- [SHINE: Syntax-augmented Hierarchical Interactive Encoder for Zero-shot Cross-lingual Information Extraction](https://mlsanity.com/paper/2305.12389/) (2023)
- [CoLaDa: A Collaborative Label Denoising Framework for Cross-lingual Named Entity Recognition](https://mlsanity.com/paper/2305.14913/) (2023)
- [2M-NER: Contrastive Learning for Multilingual and Multimodal NER with Language and Modal Fusion](https://mlsanity.com/paper/2404.17122/) (2024)

## Abstract (arXiv)

Zero-resource named entity recognition (NER) severely suffers from data scarcity in a specific domain or language. Most studies on zero-resource NER transfer knowledge from various data by fine-tuning on different auxiliary tasks. However, how to properly select training data and fine-tuning tasks is still an open problem. In this paper, we tackle the problem by transferring knowledge from three aspects, i.e., domain, language and task, and strengthening connections among them. Specifically, we propose four practical guidelines to guide knowledge transfer and task fine-tuning. Based on these guidelines, we design a target-oriented fine-tuning (TOF) framework to exploit various data from three aspects in a unified training manner. Experimental results on six benchmarks show that our method yields consistent improvements over baselines in both cross-domain and cross-lingual scenarios. Particularly, we achieve new state-of-the-art performance on five benchmarks.