# Transfer Learning for Sequence Labeling Using Source Model and Target Data

arXiv: https://arxiv.org/abs/1902.05309 · HTML: https://mlsanity.com/paper/1902.05309/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Feb 2019.

## Built on by (most cited first, 8 of 8)

- [AdaRNN: Adaptive Learning and Forecasting of Time Series](https://mlsanity.com/paper/2108.04443/) (2021)
- [Searching for Optimal Subword Tokenization in Cross-domain NER](https://mlsanity.com/paper/2206.03352/) (2022)
- [A Neural Span-Based Continual Named Entity Recognition Model](https://mlsanity.com/paper/2302.12200/) (2023)
- [A Survey of Ontology Expansion for Conversational Understanding](https://mlsanity.com/paper/2410.15019/) (2024)
- [Novel Slot Detection: A Benchmark for Discovering Unknown Slot Types in the Task-Oriented Dialogue System](https://mlsanity.com/paper/2105.14313/) (2021)
- [ProtoNER: Few shot Incremental Learning for Named Entity Recognition using Prototypical Networks](https://mlsanity.com/paper/2310.02372/) (2023)
- [Maintaining Performance with Less Data](https://mlsanity.com/paper/2208.02007/) (2022)
- [Taxonomy Expansion for Named Entity Recognition](https://mlsanity.com/paper/2305.13191/) (2023)

## Abstract (arXiv)

In this paper, we propose an approach for transferring the knowledge of a neural model for sequence labeling, learned from the source domain, to a new model trained on a target domain, where new label categories appear. Our transfer learning (TL) techniques enable to adapt the source model using the target data and new categories, without accessing to the source data. Our solution consists in adding new neurons in the output layer of the target model and transferring parameters from the source model, which are then fine-tuned with the target data. Additionally, we propose a neural adapter to learn the difference between the source and the target label distribution, which provides additional important information to the target model. Our experiments on Named Entity Recognition show that (i) the learned knowledge in the source model can be effectively transferred when the target data contains new categories and (ii) our neural adapter further improves such transfer.