# Predicting Target Language CCG Supertags Improves Neural Machine Translation

arXiv: https://arxiv.org/abs/1702.01147 · HTML: https://mlsanity.com/paper/1702.01147/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Feb 2017.

## Built on by (most cited first, 15 of 15)

- [Survey of Low-Resource Machine Translation](https://mlsanity.com/paper/2109.00486/) (2021)
- [Transformer Grammars: Augmenting Transformer Language Models with Syntactic Inductive Biases at Scale](https://mlsanity.com/paper/2203.00633/) (2022)
- [Can LLMs Really Learn to Translate a Low-Resource Language from One Grammar Book?](https://mlsanity.com/paper/2409.19151/) (2024)
- [Isochrony-Aware Neural Machine Translation for Automatic Dubbing](https://mlsanity.com/paper/2112.08548/) (2021)
- [The Role of Syntactic Planning in Compositional Image Captioning](https://mlsanity.com/paper/2101.11911/) (2021)
- [Does Entity Abstraction Help Generative Transformers Reason?](https://mlsanity.com/paper/2201.01787/) (2022)
- [Improving Isochronous Machine Translation with Target Factors and Auxiliary Counters](https://mlsanity.com/paper/2305.13204/) (2023)
- [Neural Machine Translation with Dynamic Graph Convolutional Decoder](https://mlsanity.com/paper/2305.17698/) (2023)
- [Faithful Target Attribute Prediction in Neural Machine Translation](https://mlsanity.com/paper/2109.12105/) (2021)
- [SynJax: Structured Probability Distributions for JAX](https://mlsanity.com/paper/2308.03291/) (2023)
- [Enhancing the Transformer Decoder with Transition-based Syntax](https://mlsanity.com/paper/2101.12640/) (2021)
- [Modeling Target-Side Morphology in Neural Machine Translation: A Comparison of Strategies](https://mlsanity.com/paper/2203.13550/) (2022)
- [Visualizing the Relationship Between Encoded Linguistic Information and Task Performance](https://mlsanity.com/paper/2203.15860/) (2022)
- [Understanding the effects of word-level linguistic annotations in under-resourced neural machine translation](https://mlsanity.com/paper/2401.16078/) (2024)
- [Top-down string-to-dependency Neural Machine Translation](https://mlsanity.com/paper/2603.27938/) (2026)

## Abstract (arXiv)

Neural machine translation (NMT) models are able to partially learn syntactic information from sequential lexical information. Still, some complex syntactic phenomena such as prepositional phrase attachment are poorly modeled. This work aims to answer two questions: 1) Does explicitly modeling target language syntax help NMT? 2) Is tight integration of words and syntax better than multitask training? We introduce syntactic information in the form of CCG supertags in the decoder, by interleaving the target supertags with the word sequence. Our results on WMT data show that explicitly modeling target-syntax improves machine translation quality for German->English, a high-resource pair, and for Romanian->English, a low-resource pair and also several syntactic phenomena including prepositional phrase attachment. Furthermore, a tight coupling of words and syntax improves translation quality more than multitask training. By combining target-syntax with adding source-side dependency labels in the embedding layer, we obtain a total improvement of 0.9 BLEU for German->English and 1.2 BLEU for Romanian->English.