# Grammar as a Foreign Language

arXiv: https://arxiv.org/abs/1412.7449 · HTML: https://mlsanity.com/paper/1412.7449/

## Key facts

- Cited by 70 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Dec 2014.

## Built on by (most cited first, 30 of 70)

- [Pix2seq: A Language Modeling Framework for Object Detection](https://mlsanity.com/paper/2109.10852/) (2021)
- [Grammar-Constrained Decoding for Structured NLP Tasks without Finetuning](https://mlsanity.com/paper/2305.13971/) (2023)
- [Structured Prediction as Translation between Augmented Natural Languages](https://mlsanity.com/paper/2101.05779/) (2021)
- [Episodic Transformer for Vision-and-Language Navigation](https://mlsanity.com/paper/2105.06453/) (2021)
- [Text2Event: Controllable Sequence-to-Structure Generation for End-to-end Event Extraction](https://mlsanity.com/paper/2106.09232/) (2021)
- [How to represent part-whole hierarchies in a neural network](https://mlsanity.com/paper/2102.12627/) (2021)
- [DyVal: Dynamic Evaluation of Large Language Models for Reasoning Tasks](https://mlsanity.com/paper/2309.17167/) (2023)
- [Attention, please! A survey of Neural Attention Models in Deep Learning](https://mlsanity.com/paper/2103.16775/) (2021)
- [Autoregressive Structured Prediction with Language Models](https://mlsanity.com/paper/2210.14698/) (2022)
- [Transformer Grammars: Augmenting Transformer Language Models with Syntactic Inductive Biases at Scale](https://mlsanity.com/paper/2203.00633/) (2022)
- [Paradigm Shift in Natural Language Processing](https://mlsanity.com/paper/2109.12575/) (2021)
- [Maximum Bayes Smatch Ensemble Distillation for AMR Parsing](https://mlsanity.com/paper/2112.07790/) (2021)
- [Structure-aware Fine-tuning of Sequence-to-sequence Transformers for Transition-based AMR Parsing](https://mlsanity.com/paper/2110.15534/) (2021)
- [Emotion-aware Chat Machine: Automatic Emotional Response Generation for Human-like Emotional Interaction](https://mlsanity.com/paper/2106.03044/) (2021)
- [Bottom-Up Constituency Parsing and Nested Named Entity Recognition with Pointer Networks](https://mlsanity.com/paper/2110.05419/) (2021)
- [Solving Aspect Category Sentiment Analysis as a Text Generation Task](https://mlsanity.com/paper/2110.07310/) (2021)
- [Pushdown Layers: Encoding Recursive Structure in Transformer Language Models](https://mlsanity.com/paper/2310.19089/) (2023)
- [Wasserstein Generative Adversarial Uncertainty Quantification in Physics-Informed Neural Networks](https://mlsanity.com/paper/2108.13054/) (2021)
- [Constrained Sequence-to-Tree Generation for Hierarchical Text Classification](https://mlsanity.com/paper/2204.00811/) (2022)
- [BenchCLAMP: A Benchmark for Evaluating Language Models on Syntactic and Semantic Parsing](https://mlsanity.com/paper/2206.10668/) (2022)
- [SVD-PINNs: Transfer Learning of Physics-Informed Neural Networks via Singular Value Decomposition](https://mlsanity.com/paper/2211.08760/) (2022)
- [Multistage Collaborative Knowledge Distillation from a Large Language Model for Semi-Supervised Sequence Generation](https://mlsanity.com/paper/2311.08640/) (2023)
- [Generative Pretrained Structured Transformers: Unsupervised Syntactic Language Models at Scale](https://mlsanity.com/paper/2403.08293/) (2024)
- [El Volumen Louder Por Favor: Code-switching in Task-oriented Semantic Parsing](https://mlsanity.com/paper/2101.10524/) (2021)
- [A Conditional Splitting Framework for Efficient Constituency Parsing](https://mlsanity.com/paper/2106.15760/) (2021)
- [Deep Autoregressive Models as Causal Inference Engines](https://mlsanity.com/paper/2409.18581/) (2024)
- [The Limitations of Limited Context for Constituency Parsing](https://mlsanity.com/paper/2106.01580/) (2021)
- [HPE:Answering Complex Questions over Text by Hybrid Question Parsing and Execution](https://mlsanity.com/paper/2305.07789/) (2023)
- [User-Aware Prefix-Tuning is a Good Learner for Personalized Image Captioning](https://mlsanity.com/paper/2312.04793/) (2023)
- [Discontinuous Grammar as a Foreign Language](https://mlsanity.com/paper/2110.10431/) (2021)

## Abstract (arXiv)

Syntactic constituency parsing is a fundamental problem in natural language processing and has been the subject of intensive research and engineering for decades. As a result, the most accurate parsers are domain specific, complex, and inefficient. In this paper we show that the domain agnostic attention-enhanced sequence-to-sequence model achieves state-of-the-art results on the most widely used syntactic constituency parsing dataset, when trained on a large synthetic corpus that was annotated using existing parsers. It also matches the performance of standard parsers when trained only on a small human-annotated dataset, which shows that this model is highly data-efficient, in contrast to sequence-to-sequence models without the attention mechanism. Our parser is also fast, processing over a hundred sentences per second with an unoptimized CPU implementation.