# Improved Neural Machine Translation with a Syntax-Aware Encoder and Decoder

arXiv: https://arxiv.org/abs/1707.05436 · HTML: https://mlsanity.com/paper/1707.05436/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Jul 2017.

## Built on by (most cited first, 16 of 16)

- [Learning Syntax Without Planting Trees: Understanding Hierarchical Generalization in Transformers](https://mlsanity.com/paper/2404.16367/) (2024)
- [Syntax-augmented Multilingual BERT for Cross-lingual Transfer](https://mlsanity.com/paper/2106.02134/) (2021)
- [RST-LoRA: A Discourse-Aware Low-Rank Adaptation for Long Document Abstractive Summarization](https://mlsanity.com/paper/2405.00657/) (2024)
- [Graph-Induced Syntactic-Semantic Spaces in Transformer-Based Variational AutoEncoders](https://mlsanity.com/paper/2311.08579/) (2023)
- [Transformers as Graph-to-Graph Models](https://mlsanity.com/paper/2310.17936/) (2023)
- [On the Relation between Syntactic Divergence and Zero-Shot Performance](https://mlsanity.com/paper/2110.04644/) (2021)
- [Structured Dialogue Discourse Parsing](https://mlsanity.com/paper/2306.15103/) (2023)
- [Syntax-guided Localized Self-attention by Constituency Syntactic Distance](https://mlsanity.com/paper/2210.11759/) (2022)
- [Neural Machine Translation with Dynamic Graph Convolutional Decoder](https://mlsanity.com/paper/2305.17698/) (2023)
- [Tree Transformers are an Ineffective Model of Syntactic Constituency](https://mlsanity.com/paper/2411.16993/) (2024)
- [xASTNN: Improved Code Representations for Industrial Practice](https://mlsanity.com/paper/2303.07104/) (2023)
- [Deps-SAN: Neural Machine Translation with Dependency-Scaled Self-Attention Network](https://mlsanity.com/paper/2111.11707/) (2021)
- [Exploiting Rich Syntax for Better Knowledge Base Question Answering](https://mlsanity.com/paper/2107.07940/) (2021)
- [Syntax-Aware Complex-Valued Neural Machine Translation](https://mlsanity.com/paper/2307.08586/) (2023)
- [Prediction of Translation Techniques for the Translation Process](https://mlsanity.com/paper/2403.14454/) (2024)
- [Top-down string-to-dependency Neural Machine Translation](https://mlsanity.com/paper/2603.27938/) (2026)

## Abstract (arXiv)

Most neural machine translation (NMT) models are based on the sequential encoder-decoder framework, which makes no use of syntactic information. In this paper, we improve this model by explicitly incorporating source-side syntactic trees. More specifically, we propose (1) a bidirectional tree encoder which learns both sequential and tree structured representations; (2) a tree-coverage model that lets the attention depend on the source-side syntax. Experiments on Chinese-English translation demonstrate that our proposed models outperform the sequential attentional model as well as a stronger baseline with a bottom-up tree encoder and word coverage.