# Unsupervised Cross-lingual Adaptation for Sequence Tagging and Beyond

arXiv: https://arxiv.org/abs/2010.12405 · HTML: https://mlsanity.com/paper/2010.12405/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Oct 2020.

## Built on by (most cited first, 22 of 22)

- [A Survey on Aspect-Based Sentiment Analysis: Tasks, Methods, and Challenges](https://mlsanity.com/paper/2203.01054/) (2022)
- [MELM: Data Augmentation with Masked Entity Language Modeling for Low-Resource NER](https://mlsanity.com/paper/2108.13655/) (2021)
- [mCL-NER: Cross-Lingual Named Entity Recognition via Multi-view Contrastive Learning](https://mlsanity.com/paper/2308.09073/) (2023)
- [QUEACO: Borrowing Treasures from Weakly-labeled Behavior Data for Query Attribute Value Extraction](https://mlsanity.com/paper/2108.08468/) (2021)
- [ConNER: Consistency Training for Cross-lingual Named Entity Recognition](https://mlsanity.com/paper/2211.09394/) (2022)
- [Improving Self-training for Cross-lingual Named Entity Recognition with Contrastive and Prototype Learning](https://mlsanity.com/paper/2305.13628/) (2023)
- [PeerDA: Data Augmentation via Modeling Peer Relation for Span Identification Tasks](https://mlsanity.com/paper/2210.08855/) (2022)
- [CL-XABSA: Contrastive Learning for Cross-lingual Aspect-based Sentiment Analysis](https://mlsanity.com/paper/2204.00791/) (2022)
- [Learning from Multiple Noisy Augmented Data Sets for Better Cross-Lingual Spoken Language Understanding](https://mlsanity.com/paper/2109.01583/) (2021)
- [Towards Multi-Sense Cross-Lingual Alignment of Contextual Embeddings](https://mlsanity.com/paper/2103.06459/) (2021)
- [Few-shot News Recommendation via Cross-lingual Transfer](https://mlsanity.com/paper/2207.14370/) (2022)
- [Towards Robust Low-Resource Fine-Tuning with Multi-View Compressed Representations](https://mlsanity.com/paper/2211.08794/) (2022)
- [LACA: Improving Cross-lingual Aspect-Based Sentiment Analysis with LLM Data Augmentation](https://mlsanity.com/paper/2508.09515/) (2025)
- [Advancing Cross-lingual Aspect-Based Sentiment Analysis with LLMs and Constrained Decoding for Sequence-to-Sequence Models](https://mlsanity.com/paper/2508.10366/) (2025)
- [mPMR: A Multilingual Pre-trained Machine Reader at Scale](https://mlsanity.com/paper/2305.13645/) (2023)
- [Few-shot Cross-lingual Aspect-Based Sentiment Analysis with Sequence-to-Sequence Models](https://mlsanity.com/paper/2508.07866/) (2025)
- [Cross-lingual Aspect-Based Sentiment Analysis: A Survey on Tasks, Approaches, and Challenges](https://mlsanity.com/paper/2508.09516/) (2025)
- [Improving Generative Cross-lingual Aspect-Based Sentiment Analysis with Constrained Decoding](https://mlsanity.com/paper/2508.10369/) (2025)
- [Zero-shot Cross-lingual Transfer without Parallel Corpus](https://mlsanity.com/paper/2310.04726/) (2023)
- [KPC-cF: Aspect-Based Sentiment Analysis via Implicit-Feature Alignment with Corpus Filtering](https://mlsanity.com/paper/2407.00342/) (2024)
- [MSMO-ABSA: Multi-Scale and Multi-Objective Optimization for Cross-Lingual Aspect-Based Sentiment Analysis](https://mlsanity.com/paper/2502.13718/) (2025)
- [Generative Models Enhanced by Sequence Labelling and Aspect-Code Switching Improve Cross-lingual Aspect-Based Sentiment Analysis](https://mlsanity.com/paper/2608.30425/) (2026)

## Abstract (arXiv)

Cross-lingual adaptation with multilingual pre-trained language models (mPTLMs) mainly consists of two lines of works: zero-shot approach and translation-based approach, which have been studied extensively on the sequence-level tasks. We further verify the efficacy of these cross-lingual adaptation approaches by evaluating their performances on more fine-grained sequence tagging tasks. After re-examining their strengths and drawbacks, we propose a novel framework to consolidate the zero-shot approach and the translation-based approach for better adaptation performance. Instead of simply augmenting the source data with the machine-translated data, we tailor-make a warm-up mechanism to quickly update the mPTLMs with the gradients estimated on a few translated data. Then, the adaptation approach is applied to the refined parameters and the cross-lingual transfer is performed in a warm-start way. The experimental results on nine target languages demonstrate that our method is beneficial to the cross-lingual adaptation of various sequence tagging tasks.