# A Tree-based Decoder for Neural Machine Translation

arXiv: https://arxiv.org/abs/1808.09374 · HTML: https://mlsanity.com/paper/1808.09374/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Aug 2018.

## Built on by (most cited first, 18 of 18)

- [SynGEC: Syntax-Enhanced Grammatical Error Correction with a Tailored GEC-Oriented Parser](https://mlsanity.com/paper/2210.12484/) (2022)
- [Structural Information Preserving for Graph-to-Text Generation](https://mlsanity.com/paper/2102.06749/) (2021)
- [Compressing Long Context for Enhancing RAG with AMR-based Concept Distillation](https://mlsanity.com/paper/2405.03085/) (2024)
- [Discovering Non-monotonic Autoregressive Orderings with Variational Inference](https://mlsanity.com/paper/2110.15797/) (2021)
- [Constrained Sequence-to-Tree Generation for Hierarchical Text Classification](https://mlsanity.com/paper/2204.00811/) (2022)
- [KNOD: Domain Knowledge Distilled Tree Decoder for Automated Program Repair](https://mlsanity.com/paper/2302.01857/) (2023)
- [Explicit Syntactic Guidance for Neural Text Generation](https://mlsanity.com/paper/2306.11485/) (2023)
- [Investigating Non-local Features for Neural Constituency Parsing](https://mlsanity.com/paper/2109.12814/) (2021)
- [Language Model as Visual Explainer](https://mlsanity.com/paper/2412.07802/) (2024)
- [Hierarchical information matters: Text classification via tree based graph neural network](https://mlsanity.com/paper/2110.02047/) (2021)
- [Constituency Parsing using LLMs](https://mlsanity.com/paper/2310.19462/) (2023)
- [Enhancing the Transformer Decoder with Transition-based Syntax](https://mlsanity.com/paper/2101.12640/) (2021)
- [Distilling Specialized Orders for Visual Generation](https://mlsanity.com/paper/2504.17069/) (2025)
- [To Understand Representation of Layer-aware Sequence Encoders as Multi-order-graph](https://mlsanity.com/paper/2101.06397/) (2021)
- [Understanding the effects of word-level linguistic annotations in under-resourced neural machine translation](https://mlsanity.com/paper/2401.16078/) (2024)
- [CoCR-RAG: Enhancing Retrieval-Augmented Generation in Web Q&A via Concept-oriented Context Reconstruction](https://mlsanity.com/paper/2603.23989/) (2026)
- [Top-down string-to-dependency Neural Machine Translation](https://mlsanity.com/paper/2603.27938/) (2026)
- [Exploiting Pre-trained Encoder-Decoder Transformers for Sequence-to-Sequence Constituent Parsing](https://mlsanity.com/paper/2605.13373/) (2026)

## Abstract (arXiv)

Recent advances in Neural Machine Translation (NMT) show that adding syntactic information to NMT systems can improve the quality of their translations. Most existing work utilizes some specific types of linguistically-inspired tree structures, like constituency and dependency parse trees. This is often done via a standard RNN decoder that operates on a linearized target tree structure. However, it is an open question of what specific linguistic formalism, if any, is the best structural representation for NMT. In this paper, we (1) propose an NMT model that can naturally generate the topology of an arbitrary tree structure on the target side, and (2) experiment with various target tree structures. Our experiments show the surprising result that our model delivers the best improvements with balanced binary trees constructed without any linguistic knowledge; this model outperforms standard seq2seq models by up to 2.1 BLEU points, and other methods for incorporating target-side syntax by up to 0.7 BLEU.