# Linguistic Input Features Improve Neural Machine Translation

arXiv: https://arxiv.org/abs/1606.02892 · HTML: https://mlsanity.com/paper/1606.02892/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Jun 2016.

## Built on by (most cited first, 30 of 38)

- [Survey of Low-Resource Machine Translation](https://mlsanity.com/paper/2109.00486/) (2021)
- [Domain Adaptation and Multi-Domain Adaptation for Neural Machine Translation: A Survey](https://mlsanity.com/paper/2104.06951/) (2021)
- [On the Domain Adaptation and Generalization of Pretrained Language Models: A Survey](https://mlsanity.com/paper/2211.03154/) (2022)
- [Facilitating Terminology Translation with Target Lemma Annotations](https://mlsanity.com/paper/2101.10035/) (2021)
- [Isochrony-Aware Neural Machine Translation for Automatic Dubbing](https://mlsanity.com/paper/2112.08548/) (2021)
- [Beyond Noise: Mitigating the Impact of Fine-grained Semantic Divergences on Neural Machine Translation](https://mlsanity.com/paper/2105.15087/) (2021)
- [Rethinking Data Augmentation for Low-Resource Neural Machine Translation: A Multi-Task Learning Approach](https://mlsanity.com/paper/2109.03645/) (2021)
- [Extremely low-resource machine translation for closely related languages](https://mlsanity.com/paper/2105.13065/) (2021)
- [Exploiting Dialect Identification in Automatic Dialectal Text Normalization](https://mlsanity.com/paper/2407.03020/) (2024)
- [Better Neural Machine Translation by Extracting Linguistic Information from BERT](https://mlsanity.com/paper/2104.02831/) (2021)
- [Abusive Language Detection in Heterogeneous Contexts: Dataset Collection and the Role of Supervised Attention](https://mlsanity.com/paper/2105.11119/) (2021)
- [Structural Biases for Improving Transformers on Translation into Morphologically Rich Languages](https://mlsanity.com/paper/2208.06061/) (2022)
- [Adam Mickiewicz University at WMT 2022: NER-Assisted and Quality-Aware Neural Machine Translation](https://mlsanity.com/paper/2209.02962/) (2022)
- [Variance-Aware Machine Translation Test Sets](https://mlsanity.com/paper/2111.04079/) (2021)
- [Sockeye 3: Fast Neural Machine Translation with PyTorch](https://mlsanity.com/paper/2207.05851/) (2022)
- [Diversifying Neural Text Generation with Part-of-Speech Guided Softmax and Sampling](https://mlsanity.com/paper/2105.03641/) (2021)
- [Faithful Target Attribute Prediction in Neural Machine Translation](https://mlsanity.com/paper/2109.12105/) (2021)
- [How do lexical semantics affect translation? An empirical study](https://mlsanity.com/paper/2201.00075/) (2021)
- [Learning with Latent Structures in Natural Language Processing: A Survey](https://mlsanity.com/paper/2201.00490/) (2022)
- [Improving English to Sinhala Neural Machine Translation using Part-of-Speech Tag](https://mlsanity.com/paper/2202.08882/) (2022)
- [Context-Aware Machine Translation with Source Coreference Explanation](https://mlsanity.com/paper/2404.19505/) (2024)
- [Treasure Hunt: Real-time Targeting of the Long Tail using Training-Time Markers](https://mlsanity.com/paper/2506.14702/) (2025)
- [Deps-SAN: Neural Machine Translation with Dependency-Scaled Self-Attention Network](https://mlsanity.com/paper/2111.11707/) (2021)
- [Linguistically-driven Multi-task Pre-training for Low-resource Neural Machine Translation](https://mlsanity.com/paper/2201.08070/) (2022)
- [Visualizing the Relationship Between Encoded Linguistic Information and Task Performance](https://mlsanity.com/paper/2203.15860/) (2022)
- [Synergizing Machine Learning & Symbolic Methods: A Survey on Hybrid Approaches to Natural Language Processing](https://mlsanity.com/paper/2401.11972/) (2024)
- [Inducing Systematicity in Transformers by Attending to Structurally Quantized Embeddings](https://mlsanity.com/paper/2402.06492/) (2024)
- [PHRASED: Phrase Dictionary Biasing for Speech Translation](https://mlsanity.com/paper/2506.09175/) (2025)
- [Open Your Model's Eyes: Video and Context-Aware Multimodal Backchannel Prediction](https://mlsanity.com/paper/2607.22729/) (2026)
- [Sparsely Factored Neural Machine Translation](https://mlsanity.com/paper/2102.08934/) (2021)

## Abstract (arXiv)

Neural machine translation has recently achieved impressive results, while using little in the way of external linguistic information. In this paper we show that the strong learning capability of neural MT models does not make linguistic features redundant; they can be easily incorporated to provide further improvements in performance. We generalize the embedding layer of the encoder in the attentional encoder--decoder architecture to support the inclusion of arbitrary features, in addition to the baseline word feature. We add morphological features, part-of-speech tags, and syntactic dependency labels as input features to English<->German, and English->Romanian neural machine translation systems. In experiments on WMT16 training and test sets, we find that linguistic input features improve model quality according to three metrics: perplexity, BLEU and CHRF3. An open-source implementation of our neural MT system is available, as are sample files and configurations.