# Adversarial Propagation and Zero-Shot Cross-Lingual Transfer of Word Vector Specialization

arXiv: https://arxiv.org/abs/1809.04163 · HTML: https://mlsanity.com/paper/1809.04163/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Sep 2018.

## Built on by (most cited first, 12 of 12)

- [Language Models are Few-shot Multilingual Learners](https://mlsanity.com/paper/2109.07684/) (2021)
- [Crossing the Conversational Chasm: A Primer on Natural Language Processing for Multilingual Task-Oriented Dialogue Systems](https://mlsanity.com/paper/2104.08570/) (2021)
- [Contextual Semantic Parsing for Multilingual Task-Oriented Dialogues](https://mlsanity.com/paper/2111.02574/) (2021)
- [CD2CR: Co-reference Resolution Across Documents and Domains](https://mlsanity.com/paper/2101.12637/) (2021)
- [Fine-Tuning BERT with Character-Level Noise for Zero-Shot Transfer to Dialects and Closely-Related Languages](https://mlsanity.com/paper/2303.17683/) (2023)
- [SenWiCh: Sense-Annotation of Low-Resource Languages for WiC using Hybrid Methods](https://mlsanity.com/paper/2505.23714/) (2025)
- [RetroGAN: A Cyclic Post-Specialization System for Improving Out-of-Knowledge and Rare Word Representations](https://mlsanity.com/paper/2108.12941/) (2021)
- [Evaluation of taxonomic and neural embedding methods for calculating semantic similarity](https://mlsanity.com/paper/2209.15197/) (2022)
- [SexWEs: Domain-Aware Word Embeddings via Cross-lingual Semantic Specialisation for Chinese Sexism Detection in Social Media](https://mlsanity.com/paper/2211.08447/) (2022)
- [Adversarial Transformer Language Models for Contextual Commonsense Inference](https://mlsanity.com/paper/2302.05406/) (2023)
- [Massively Multilingual Lexical Specialization of Multilingual Transformers](https://mlsanity.com/paper/2208.01018/) (2022)
- [Multilinguality Does not Make Sense: Investigating Factors Behind Zero-Shot Transfer in Sense-Aware Tasks](https://mlsanity.com/paper/2505.24834/) (2025)

## Abstract (arXiv)

Semantic specialization is the process of fine-tuning pre-trained distributional word vectors using external lexical knowledge (e.g., WordNet) to accentuate a particular semantic relation in the specialized vector space. While post-processing specialization methods are applicable to arbitrary distributional vectors, they are limited to updating only the vectors of words occurring in external lexicons (i.e., seen words), leaving the vectors of all other words unchanged. We propose a novel approach to specializing the full distributional vocabulary. Our adversarial post-specialization method propagates the external lexical knowledge to the full distributional space. We exploit words seen in the resources as training examples for learning a global specialization function. This function is learned by combining a standard L2-distance loss with an adversarial loss: the adversarial component produces more realistic output vectors. We show the effectiveness and robustness of the proposed method across three languages and on three tasks: word similarity, dialog state tracking, and lexical simplification. We report consistent improvements over distributional word vectors and vectors specialized by other state-of-the-art specialization frameworks. Finally, we also propose a cross-lingual transfer method for zero-shot specialization which successfully specializes a full target distributional space without any lexical knowledge in the target language and without any bilingual data.