# Shared-Private Bilingual Word Embeddings for Neural Machine Translation

arXiv: https://arxiv.org/abs/1906.03100 · HTML: https://mlsanity.com/paper/1906.03100/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Jun 2019.

## Built on by (most cited first, 9 of 9)

- [Meta-Curriculum Learning for Domain Adaptation in Neural Machine Translation](https://mlsanity.com/paper/2103.02262/) (2021)
- [Breaking the Representation Bottleneck of Chinese Characters: Neural Machine Translation with Stroke Sequence Modeling](https://mlsanity.com/paper/2211.12781/) (2022)
- [Unsupervised Document Embedding via Contrastive Augmentation](https://mlsanity.com/paper/2103.14542/) (2021)
- [ConsistTL: Modeling Consistency in Transfer Learning for Low-Resource Neural Machine Translation](https://mlsanity.com/paper/2212.04262/) (2022)
- [3AM: An Ambiguity-Aware Multi-Modal Machine Translation Dataset](https://mlsanity.com/paper/2404.18413/) (2024)
- [Variance-Aware Machine Translation Test Sets](https://mlsanity.com/paper/2111.04079/) (2021)
- [Focus on the Target's Vocabulary: Masked Label Smoothing for Machine Translation](https://mlsanity.com/paper/2203.02889/) (2022)
- [Understanding Privacy Risks of Embeddings Induced by Large Language Models](https://mlsanity.com/paper/2404.16587/) (2024)
- [Refining the state-of-the-art in Machine Translation, optimizing NMT for the JA <-> EN language pair by leveraging personal domain expertise](https://mlsanity.com/paper/2202.11669/) (2022)

## Abstract (arXiv)

Word embedding is central to neural machine translation (NMT), which has attracted intensive research interest in recent years. In NMT, the source embedding plays the role of the entrance while the target embedding acts as the terminal. These layers occupy most of the model parameters for representation learning. Furthermore, they indirectly interface via a soft-attention mechanism, which makes them comparatively isolated. In this paper, we propose shared-private bilingual word embeddings, which give a closer relationship between the source and target embeddings, and which also reduce the number of model parameters. For similar source and target words, their embeddings tend to share a part of the features and they cooperatively learn these common representation units. Experiments on 5 language pairs belonging to 6 different language families and written in 5 different alphabets demonstrate that the proposed model provides a significant performance boost over the strong baselines with dramatically fewer model parameters.