# Transfer Learning for Sequence Tagging with Hierarchical Recurrent Networks

arXiv: https://arxiv.org/abs/1703.06345 · HTML: https://mlsanity.com/paper/1703.06345/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Mar 2017.

## Built on by (most cited first, 30 of 30)

- [AdaRNN: Adaptive Learning and Forecasting of Time Series](https://mlsanity.com/paper/2108.04443/) (2021)
- [A Survey on Green Deep Learning](https://mlsanity.com/paper/2111.05193/) (2021)
- [AdaTag: Multi-Attribute Value Extraction from Product Profiles with Adaptive Decoding](https://mlsanity.com/paper/2106.02318/) (2021)
- [Neural ranking models for document retrieval](https://mlsanity.com/paper/2102.11903/) (2021)
- [Reinforced Iterative Knowledge Distillation for Cross-Lingual Named Entity Recognition](https://mlsanity.com/paper/2106.00241/) (2021)
- [METS-CoV: A Dataset of Medical Entity and Targeted Sentiment on COVID-19 Related Tweets](https://mlsanity.com/paper/2209.13773/) (2022)
- [Cross-Domain Multi-Task Learning for Sequential Sentence Classification in Research Papers](https://mlsanity.com/paper/2102.06008/) (2021)
- [Recent Trends in Named Entity Recognition (NER)](https://mlsanity.com/paper/2101.11420/) (2021)
- [Computer Science Named Entity Recognition in the Open Research Knowledge Graph](https://mlsanity.com/paper/2203.14579/) (2022)
- [FactMix: Using a Few Labeled In-domain Examples to Generalize to Cross-domain Named Entity Recognition](https://mlsanity.com/paper/2208.11464/) (2022)
- [Argument Mining in Data Scarce Settings: Cross-lingual Transfer and Few-shot Techniques](https://mlsanity.com/paper/2407.03748/) (2024)
- [Latent-Optimized Adversarial Neural Transfer for Sarcasm Detection](https://mlsanity.com/paper/2104.09261/) (2021)
- [Soft Layer Selection with Meta-Learning for Zero-Shot Cross-Lingual Transfer](https://mlsanity.com/paper/2107.09840/) (2021)
- [Transfer learning for conflict and duplicate detection in software requirement pairs](https://mlsanity.com/paper/2301.03709/) (2023)
- [From West to East: Who can understand the music of the others better?](https://mlsanity.com/paper/2307.09795/) (2023)
- [Cross-domain Named Entity Recognition via Graph Matching](https://mlsanity.com/paper/2408.00981/) (2024)
- [DyLex: Incorporating Dynamic Lexicons into BERT for Sequence Labeling](https://mlsanity.com/paper/2109.08818/) (2021)
- [Cross-lingual Argument Mining in the Medical Domain](https://mlsanity.com/paper/2301.10527/) (2023)
- [Cross-Domain Contract Element Extraction with a Bi-directional Feedback Clause-Element Relation Network](https://mlsanity.com/paper/2105.06083/) (2021)
- [Low-Resource Named Entity Recognition with Cross-Lingual, Character-Level Neural Conditional Random Fields](https://mlsanity.com/paper/2404.09383/) (2024)
- [A Robust and Domain-Adaptive Approach for Low-Resource Named Entity Recognition](https://mlsanity.com/paper/2101.00388/) (2021)
- [A Dual-Contrastive Framework for Low-Resource Cross-Lingual Named Entity Recognition](https://mlsanity.com/paper/2204.00796/) (2022)
- [Cross-domain Generalization for AMR Parsing](https://mlsanity.com/paper/2210.12445/) (2022)
- [All Data on the Table: Novel Dataset and Benchmark for Cross-Modality Scientific Information Extraction](https://mlsanity.com/paper/2311.08189/) (2023)
- [SGPT: Semantic Graphs based Pre-training for Aspect-based Sentiment Analysis](https://mlsanity.com/paper/2105.12305/) (2021)
- [Software Entity Recognition with Noise-Robust Learning](https://mlsanity.com/paper/2308.10564/) (2023)
- [Exploiting Global Contextual Information for Document-level Named Entity Recognition](https://mlsanity.com/paper/2106.00887/) (2021)
- [Using Domain Knowledge for Low Resource Named Entity Recognition](https://mlsanity.com/paper/2203.14738/) (2022)
- [A Neural Network Based Method with Transfer Learning for Genetic Data Analysis](https://mlsanity.com/paper/2206.09872/) (2022)
- [Reference Resolution and Context Change in Multimodal Situated Dialogue for Exploring Data Visualizations](https://mlsanity.com/paper/2209.02215/) (2022)

## Abstract (arXiv)

Recent papers have shown that neural networks obtain state-of-the-art performance on several different sequence tagging tasks. One appealing property of such systems is their generality, as excellent performance can be achieved with a unified architecture and without task-specific feature engineering. However, it is unclear if such systems can be used for tasks without large amounts of training data. In this paper we explore the problem of transfer learning for neural sequence taggers, where a source task with plentiful annotations (e.g., POS tagging on Penn Treebank) is used to improve performance on a target task with fewer available annotations (e.g., POS tagging for microblogs). We examine the effects of transfer learning for deep hierarchical recurrent networks across domains, applications, and languages, and show that significant improvement can often be obtained. These improvements lead to improvements over the current state-of-the-art on several well-studied tasks.