# Multi-Task Cross-Lingual Sequence Tagging from Scratch

arXiv: https://arxiv.org/abs/1603.06270 · HTML: https://mlsanity.com/paper/1603.06270/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Mar 2016.

## Built on by (most cited first, 6 of 6)

- [DeepStruct: Pretraining of Language Models for Structure Prediction](https://mlsanity.com/paper/2205.10475/) (2022)
- [Why Does Zero-Shot Cross-Lingual Generation Fail? An Explanation and a Solution](https://mlsanity.com/paper/2305.17325/) (2023)
- [Recent Trends in Named Entity Recognition (NER)](https://mlsanity.com/paper/2101.11420/) (2021)
- [Polyglot Prompt: Multilingual Multitask PrompTraining](https://mlsanity.com/paper/2204.14264/) (2022)
- [Effective Transfer Learning for Low-Resource Natural Language Understanding](https://mlsanity.com/paper/2208.09180/) (2022)
- [Language Independent Named Entity Recognition via Orthogonal Transformation of Word Vectors](https://mlsanity.com/paper/2503.14755/) (2025)

## Abstract (arXiv)

We present a deep hierarchical recurrent neural network for sequence tagging. Given a sequence of words, our model employs deep gated recurrent units on both character and word levels to encode morphology and context information, and applies a conditional random field layer to predict the tags. Our model is task independent, language independent, and feature engineering free. We further extend our model to multi-task and cross-lingual joint training by sharing the architecture and parameters. Our model achieves state-of-the-art results in multiple languages on several benchmark tasks including POS tagging, chunking, and NER. We also demonstrate that multi-task and cross-lingual joint training can improve the performance in various cases.