# Multilingual Models for Compositional Distributed Semantics

arXiv: https://arxiv.org/abs/1404.4641 · HTML: https://mlsanity.com/paper/1404.4641/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Apr 2014.

## Built on by (most cited first, 18 of 18)

- [Are Multilingual Models Effective in Code-Switching?](https://mlsanity.com/paper/2103.13309/) (2021)
- [XeroAlign: Zero-Shot Cross-lingual Transformer Alignment](https://mlsanity.com/paper/2105.02472/) (2021)
- [Crosslingual Transfer Learning for Low-Resource Languages Based on Multilingual Colexification Graphs](https://mlsanity.com/paper/2305.12818/) (2023)
- [Enhancing Cross-lingual Natural Language Inference by Soft Prompting with Multilingual Verbalizer](https://mlsanity.com/paper/2305.12761/) (2023)
- [HIT: A Hierarchically Fused Deep Attention Network for Robust Code-mixed Language Representation](https://mlsanity.com/paper/2105.14600/) (2021)
- [Retrofitting Multilingual Sentence Embeddings with Abstract Meaning Representation](https://mlsanity.com/paper/2210.09773/) (2022)
- [Beyond Contrastive Learning: A Variational Generative Model for Multilingual Retrieval](https://mlsanity.com/paper/2212.10726/) (2022)
- [Sentence Alignment with Parallel Documents Facilitates Biomedical Machine Translation](https://mlsanity.com/paper/2104.08588/) (2021)
- [A Multilingual Bag-of-Entities Model for Zero-Shot Cross-Lingual Text Classification](https://mlsanity.com/paper/2110.07792/) (2021)
- [A Comprehensive Understanding of Code-mixed Language Semantics using Hierarchical Transformer](https://mlsanity.com/paper/2204.12753/) (2022)
- [MoSECroT: Model Stitching with Static Word Embeddings for Crosslingual Zero-shot Transfer](https://mlsanity.com/paper/2401.04821/) (2024)
- [Character-level Chinese Backpack Language Models](https://mlsanity.com/paper/2310.12751/) (2023)
- [Computational Approaches to Arabic-English Code-Switching](https://mlsanity.com/paper/2410.13318/) (2024)
- [Bilingual Language Modeling, A transfer learning technique for Roman Urdu](https://mlsanity.com/paper/2102.10958/) (2021)
- [Sentiment Classification of Code-Switched Text using Pre-trained Multilingual Embeddings and Segmentation](https://mlsanity.com/paper/2210.16461/) (2022)
- [Speech representation learning: Learning bidirectional encoders with single-view, multi-view, and multi-task methods](https://mlsanity.com/paper/2308.00129/) (2023)
- [Cross-Lingual Word Alignment for ASEAN Languages with Contrastive Learning](https://mlsanity.com/paper/2407.05054/) (2024)
- [Corpus-Based Approaches to Igbo Diacritic Restoration](https://mlsanity.com/paper/2601.18380/) (2026)

## Abstract (arXiv)

We present a novel technique for learning semantic representations, which extends the distributional hypothesis to multilingual data and joint-space embeddings. Our models leverage parallel data and learn to strongly align the embeddings of semantically equivalent sentences, while maintaining sufficient distance between those of dissimilar sentences. The models do not rely on word alignments or any syntactic information and are successfully applied to a number of diverse languages. We extend our approach to learn semantic representations at the document level, too. We evaluate these models on two cross-lingual document classification tasks, outperforming the prior state of the art. Through qualitative analysis and the study of pivoting effects we demonstrate that our representations are semantically plausible and can capture semantic relationships across languages without parallel data.