# Semantic Specialisation of Distributional Word Vector Spaces using Monolingual and Cross-Lingual Constraints

arXiv: https://arxiv.org/abs/1706.00374 · HTML: https://mlsanity.com/paper/1706.00374/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Jun 2017.

## Built on by (most cited first, 20 of 20)

- [Multi2WOZ: A Robust Multilingual Dataset and Conversational Pretraining for Task-Oriented Dialog](https://mlsanity.com/paper/2205.10400/) (2022)
- [Survey of Social Bias in Vision-Language Models](https://mlsanity.com/paper/2309.14381/) (2023)
- [ConvFiT: Conversational Fine-Tuning of Pretrained Language Models](https://mlsanity.com/paper/2109.10126/) (2021)
- [Crossing the Conversational Chasm: A Primer on Natural Language Processing for Multilingual Task-Oriented Dialogue Systems](https://mlsanity.com/paper/2104.08570/) (2021)
- [Multi3WOZ: A Multilingual, Multi-Domain, Multi-Parallel Dataset for Training and Evaluating Culturally Adapted Task-Oriented Dialog Systems](https://mlsanity.com/paper/2307.14031/) (2023)
- [Specialized Document Embeddings for Aspect-based Similarity of Research Papers](https://mlsanity.com/paper/2203.14541/) (2022)
- [Embracing Ambiguity: Improving Similarity-oriented Tasks with Contextual Synonym Knowledge](https://mlsanity.com/paper/2211.10997/) (2022)
- [Cross-lingual Intermediate Fine-tuning improves Dialogue State Tracking](https://mlsanity.com/paper/2109.13620/) (2021)
- [AllWOZ: Towards Multilingual Task-Oriented Dialog Systems for All](https://mlsanity.com/paper/2112.08333/) (2021)
- [Zero and Few-Shot Localization of Task-Oriented Dialogue Agents with a Distilled Representation](https://mlsanity.com/paper/2302.09424/) (2023)
- [RetroGAN: A Cyclic Post-Specialization System for Improving Out-of-Knowledge and Rare Word Representations](https://mlsanity.com/paper/2108.12941/) (2021)
- [An Empirical Study of Cross-Lingual Transferability in Generative Dialogue State Tracker](https://mlsanity.com/paper/2101.11360/) (2021)
- [Evaluation of taxonomic and neural embedding methods for calculating semantic similarity](https://mlsanity.com/paper/2209.15197/) (2022)
- [SexWEs: Domain-Aware Word Embeddings via Cross-lingual Semantic Specialisation for Chinese Sexism Detection in Social Media](https://mlsanity.com/paper/2211.08447/) (2022)
- [Semantic Specialization for Knowledge-based Word Sense Disambiguation](https://mlsanity.com/paper/2304.11340/) (2023)
- [Retrofitting Light-weight Language Models for Emotions using Supervised Contrastive Learning](https://mlsanity.com/paper/2310.18930/) (2023)
- [Lexical semantics enhanced neural word embeddings](https://mlsanity.com/paper/2210.00754/) (2022)
- [Analyzing Correlations Between Intrinsic and Extrinsic Bias Metrics of Static Word Embeddings With Their Measuring Biases Aligned](https://mlsanity.com/paper/2409.09260/) (2024)
- [ViWOZ: A Multi-Domain Task-Oriented Dialogue Systems Dataset For Low-resource Language](https://mlsanity.com/paper/2203.07742/) (2022)
- [Massively Multilingual Lexical Specialization of Multilingual Transformers](https://mlsanity.com/paper/2208.01018/) (2022)

## Abstract (arXiv)

We present Attract-Repel, an algorithm for improving the semantic quality of word vectors by injecting constraints extracted from lexical resources. Attract-Repel facilitates the use of constraints from mono- and cross-lingual resources, yielding semantically specialised cross-lingual vector spaces. Our evaluation shows that the method can make use of existing cross-lingual lexicons to construct high-quality vector spaces for a plethora of different languages, facilitating semantic transfer from high- to lower-resource ones. The effectiveness of our approach is demonstrated with state-of-the-art results on semantic similarity datasets in six languages. We next show that Attract-Repel-specialised vectors boost performance in the downstream task of dialogue state tracking (DST) across multiple languages. Finally, we show that cross-lingual vector spaces produced by our algorithm facilitate the training of multilingual DST models, which brings further performance improvements.