# Adversarial Learning with Contextual Embeddings for Zero-resource Cross-lingual Classification and NER

arXiv: https://arxiv.org/abs/1909.00153 · HTML: https://mlsanity.com/paper/1909.00153/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Aug 2019.

## Built on by (most cited first, 25 of 25)

- [Enhancing Cross-lingual Transfer by Manifold Mixup](https://mlsanity.com/paper/2205.04182/) (2022)
- [English Contrastive Learning Can Learn Universal Cross-lingual Sentence Embeddings](https://mlsanity.com/paper/2211.06127/) (2022)
- [AdvPicker: Effectively Leveraging Unlabeled Data via Adversarial Discriminator for Cross-Lingual NER](https://mlsanity.com/paper/2106.02300/) (2021)
- [Contributions of Transformer Attention Heads in Multi- and Cross-lingual Tasks](https://mlsanity.com/paper/2108.08375/) (2021)
- [Everything Is All It Takes: A Multipronged Strategy for Zero-Shot Cross-Lingual Information Extraction](https://mlsanity.com/paper/2109.06798/) (2021)
- [mCL-NER: Cross-Lingual Named Entity Recognition via Multi-view Contrastive Learning](https://mlsanity.com/paper/2308.09073/) (2023)
- [Nearest Neighbour Few-Shot Learning for Cross-lingual Classification](https://mlsanity.com/paper/2109.02221/) (2021)
- [A Survey on Arabic Named Entity Recognition: Past, Recent Advances, and Future Trends](https://mlsanity.com/paper/2302.03512/) (2023)
- [Improving Self-training for Cross-lingual Named Entity Recognition with Contrastive and Prototype Learning](https://mlsanity.com/paper/2305.13628/) (2023)
- [WIDER & CLOSER: Mixture of Short-channel Distillers for Zero-shot Cross-lingual Named Entity Recognition](https://mlsanity.com/paper/2212.03506/) (2022)
- [Multilingual Sentence-T5: Scalable Sentence Encoders for Multilingual Applications](https://mlsanity.com/paper/2403.17528/) (2024)
- [Few-Shot Cross-lingual Transfer for Coarse-grained De-identification of Code-Mixed Clinical Texts](https://mlsanity.com/paper/2204.04775/) (2022)
- [Domain Adaptation in Multilingual and Multi-Domain Monolingual Settings for Complex Word Identification](https://mlsanity.com/paper/2205.07283/) (2022)
- [A Multilingual Bag-of-Entities Model for Zero-Shot Cross-Lingual Text Classification](https://mlsanity.com/paper/2110.07792/) (2021)
- [A Dual-Contrastive Framework for Low-Resource Cross-Lingual Named Entity Recognition](https://mlsanity.com/paper/2204.00796/) (2022)
- [Cross-lingual Transfer Learning for Check-worthy Claim Identification over Twitter](https://mlsanity.com/paper/2211.05087/) (2022)
- [CMNER: A Chinese Multimodal NER Dataset based on Social Media](https://mlsanity.com/paper/2402.13693/) (2024)
- [Improving Pseudo Labels with Global-Local Denoising Framework for Cross-lingual Named Entity Recognition](https://mlsanity.com/paper/2406.01213/) (2024)
- [SHINE: Syntax-augmented Hierarchical Interactive Encoder for Zero-shot Cross-lingual Information Extraction](https://mlsanity.com/paper/2305.12389/) (2023)
- [Out-of-Distribution Generalization in Text Classification: Past, Present, and Future](https://mlsanity.com/paper/2305.14104/) (2023)
- [CoLaDa: A Collaborative Label Denoising Framework for Cross-lingual Named Entity Recognition](https://mlsanity.com/paper/2305.14913/) (2023)
- [Zero-Resource Multi-Dialectal Arabic Natural Language Understanding](https://mlsanity.com/paper/2104.06591/) (2021)
- [Zero-Shot Cross-Lingual Transfer in Legal Domain Using Transformer Models](https://mlsanity.com/paper/2111.14192/) (2021)
- [Learning for Amalgamation: A Multi-Source Transfer Learning Framework For Sentiment Classification](https://mlsanity.com/paper/2303.09115/) (2023)
- [Sharing, Teaching and Aligning: Knowledgeable Transfer Learning for Cross-Lingual Machine Reading Comprehension](https://mlsanity.com/paper/2311.06758/) (2023)

## Abstract (arXiv)

Contextual word embeddings (e.g. GPT, BERT, ELMo, etc.) have demonstrated state-of-the-art performance on various NLP tasks. Recent work with the multilingual version of BERT has shown that the model performs very well in zero-shot and zero-resource cross-lingual settings, where only labeled English data is used to finetune the model. We improve upon multilingual BERT's zero-resource cross-lingual performance via adversarial learning. We report the magnitude of the improvement on the multilingual MLDoc text classification and CoNLL 2002/2003 named entity recognition tasks. Furthermore, we show that language-adversarial training encourages BERT to align the embeddings of English documents and their translations, which may be the cause of the observed performance gains.