# Syntax-Enhanced Neural Machine Translation with Syntax-Aware Word Representations

arXiv: https://arxiv.org/abs/1905.02878 · HTML: https://mlsanity.com/paper/1905.02878/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 May 2019.

## Built on by (most cited first, 10 of 10)

- [SynGEC: Syntax-Enhanced Grammatical Error Correction with a Tailored GEC-Oriented Parser](https://mlsanity.com/paper/2210.12484/) (2022)
- [FastKASSIM: A Fast Tree Kernel-Based Syntactic Similarity Metric](https://mlsanity.com/paper/2203.08299/) (2022)
- [Dependency Parsing with Bottom-up Hierarchical Pointer Networks](https://mlsanity.com/paper/2105.09611/) (2021)
- [Discontinuous Grammar as a Foreign Language](https://mlsanity.com/paper/2110.10431/) (2021)
- [Zero-Shot Cross-Lingual Machine Reading Comprehension via Inter-sentence Dependency Graph](https://mlsanity.com/paper/2112.00503/) (2021)
- [Enhancing the Transformer Decoder with Transition-based Syntax](https://mlsanity.com/paper/2101.12640/) (2021)
- [Deps-SAN: Neural Machine Translation with Dependency-Scaled Self-Attention Network](https://mlsanity.com/paper/2111.11707/) (2021)
- [To Understand Representation of Layer-aware Sequence Encoders as Multi-order-graph](https://mlsanity.com/paper/2101.06397/) (2021)
- [Combining Improvements for Exploiting Dependency Trees in Neural Semantic Parsing](https://mlsanity.com/paper/2112.13179/) (2021)
- [NLPre: a revised approach towards language-centric benchmarking of Natural Language Preprocessing systems](https://mlsanity.com/paper/2403.04507/) (2024)

## Abstract (arXiv)

Syntax has been demonstrated highly effective in neural machine translation (NMT). Previous NMT models integrate syntax by representing 1-best tree outputs from a well-trained parsing system, e.g., the representative Tree-RNN and Tree-Linearization methods, which may suffer from error propagation. In this work, we propose a novel method to integrate source-side syntax implicitly for NMT. The basic idea is to use the intermediate hidden representations of a well-trained end-to-end dependency parser, which are referred to as syntax-aware word representations (SAWRs). Then, we simply concatenate such SAWRs with ordinary word embeddings to enhance basic NMT models. The method can be straightforwardly integrated into the widely-used sequence-to-sequence (Seq2Seq) NMT models. We start with a representative RNN-based Seq2Seq baseline system, and test the effectiveness of our proposed method on two benchmark datasets of the Chinese-English and English-Vietnamese translation tasks, respectively. Experimental results show that the proposed approach is able to bring significant BLEU score improvements on the two datasets compared with the baseline, 1.74 points for Chinese-English translation and 0.80 point for English-Vietnamese translation, respectively. In addition, the approach also outperforms the explicit Tree-RNN and Tree-Linearization methods.