# Compositional Representation of Morphologically-Rich Input for Neural Machine Translation

arXiv: https://arxiv.org/abs/1805.02036 · HTML: https://mlsanity.com/paper/1805.02036/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 May 2018.

## Built on by (most cited first, 12 of 12)

- [Between words and characters: A Brief History of Open-Vocabulary Modeling and Tokenization in NLP](https://mlsanity.com/paper/2112.10508/) (2021)
- [Multi-view Subword Regularization](https://mlsanity.com/paper/2103.08490/) (2021)
- [The Effectiveness of Morphology-aware Segmentation in Low-Resource Neural Machine Translation](https://mlsanity.com/paper/2103.11189/) (2021)
- [How Suitable Are Subword Segmentation Strategies for Translating Non-Concatenative Morphology?](https://mlsanity.com/paper/2109.01100/) (2021)
- [Neural Machine Translation for the Indigenous Languages of the Americas: An Introduction](https://mlsanity.com/paper/2306.06804/) (2023)
- [Low-resource neural machine translation with morphological modeling](https://mlsanity.com/paper/2404.02392/) (2024)
- [Rule-based Morphological Inflection Improves Neural Terminology Translation](https://mlsanity.com/paper/2109.04620/) (2021)
- [Towards Equipping Transformer with the Ability of Systematic Compositionality](https://mlsanity.com/paper/2312.07280/) (2023)
- [Tokenization with Split Trees](https://mlsanity.com/paper/2605.22705/) (2026)
- [Modeling Target-Side Morphology in Neural Machine Translation: A Comparison of Strategies](https://mlsanity.com/paper/2203.13550/) (2022)
- [The boundaries of meaning: a case study in neural machine translation](https://mlsanity.com/paper/2210.00613/) (2022)
- [LGSE: Lexically Grounded Subword Embedding Initialization for Low-Resource Language Adaptation](https://mlsanity.com/paper/2603.22629/) (2026)

## Abstract (arXiv)

Neural machine translation (NMT) models are typically trained with fixed-size input and output vocabularies, which creates an important bottleneck on their accuracy and generalization capability. As a solution, various studies proposed segmenting words into sub-word units and performing translation at the sub-lexical level. However, statistical word segmentation methods have recently shown to be prone to morphological errors, which can lead to inaccurate translations. In this paper, we propose to overcome this problem by replacing the source-language embedding layer of NMT with a bi-directional recurrent neural network that generates compositional representations of the input at any desired level of granularity. We test our approach in a low-resource setting with five languages from different morphological typologies, and under different composition assumptions. By training NMT to compose word representations from character n-grams, our approach consistently outperforms (from 1.71 to 2.48 BLEU points) NMT learning embeddings of statistically generated sub-word units.