# Top-down string-to-dependency Neural Machine Translation

arXiv: https://arxiv.org/abs/2603.27938 · HTML: https://mlsanity.com/paper/2603.27938/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 28 of its 49 references that are in the map.
- First posted on arXiv 30 Mar 2026.

## Builds on (most cited first, 28 of 28)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context](https://mlsanity.com/paper/1901.02860/) (2019)
- [Effective Approaches to Attention-based Neural Machine Translation](https://mlsanity.com/paper/1508.04025/) (2015)
- [Using the Output Embedding to Improve Language Models](https://mlsanity.com/paper/1608.05859/) (2016)
- [A Theoretically Grounded Application of Dropout in Recurrent Neural Networks](https://mlsanity.com/paper/1512.05287/) (2015)
- [Recurrent Neural Network Grammars](https://mlsanity.com/paper/1602.07776/) (2016)
- [The Best of Both Worlds: Combining Recent Advances in Neural Machine Translation](https://mlsanity.com/paper/1804.09849/) (2018)
- [Tying Word Vectors and Word Classifiers: A Loss Framework for Language Modeling](https://mlsanity.com/paper/1611.01462/) (2016)
- [Transformer Grammars: Augmenting Transformer Language Models with Syntactic Inductive Biases at Scale](https://mlsanity.com/paper/2203.00633/) (2022)
- [Tree-to-Sequence Attentional Neural Machine Translation](https://mlsanity.com/paper/1603.06075/) (2016)
- [Towards String-to-Tree Neural Machine Translation](https://mlsanity.com/paper/1704.04743/) (2017)
- [Scheduled Multi-Task Learning: From Syntax to Translation](https://mlsanity.com/paper/1804.08915/) (2018)
- [A Tree-based Decoder for Neural Machine Translation](https://mlsanity.com/paper/1808.09374/) (2018)
- [Learning to Parse and Translate Improves Neural Machine Translation](https://mlsanity.com/paper/1702.03525/) (2017)
- [Improved Neural Machine Translation with a Syntax-Aware Encoder and Decoder](https://mlsanity.com/paper/1707.05436/) (2017)
- [Predicting Target Language CCG Supertags Improves Neural Machine Translation](https://mlsanity.com/paper/1702.01147/) (2017)
- [Top-down Tree Long Short-Term Memory Networks](https://mlsanity.com/paper/1511.00060/) (2015)
- [Modeling Source Syntax for Neural Machine Translation](https://mlsanity.com/paper/1705.01020/) (2017)
- [Explicit Syntactic Guidance for Neural Text Generation](https://mlsanity.com/paper/2306.11485/) (2023)
- [Dependency Transformer Grammars: Integrating Dependency Structures into Transformer Language Models](https://mlsanity.com/paper/2407.17406/) (2024)
- [Multi-representation Ensembles and Delayed SGD Updates Improve Syntax-based NMT](https://mlsanity.com/paper/1805.00456/) (2018)

## Abstract (arXiv)

Most of modern neural machine translation (NMT) models are based on an encoder-decoder framework with an attention mechanism. While they perform well on standard datasets, they can have trouble in translation of long inputs that are rare or unseen during training. Incorporating target syntax is one approach to dealing with such length-related problems. We propose a novel syntactic decoder that generates a target-language dependency tree in a top-down, left-to-right order. Experiments show that the proposed top-down string-to-tree decoding generalizes better than conventional sequence-to-sequence decoding in translating long inputs that are not observed in the training data.