# A Grammar-Based Structural CNN Decoder for Code Generation

arXiv: https://arxiv.org/abs/1811.06837 · HTML: https://mlsanity.com/paper/1811.06837/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Nov 2018.

## Built on by (most cited first, 29 of 29)

- [Large Language Models Meet NL2Code: A Survey](https://mlsanity.com/paper/2212.09420/) (2022)
- [CERT: Continual Pre-Training on Sketches for Library-Oriented Code Generation](https://mlsanity.com/paper/2206.06888/) (2022)
- [Out of the BLEU: how should we assess quality of the Code Generation models?](https://mlsanity.com/paper/2208.03133/) (2022)
- [A Syntax-Guided Edit Decoder for Neural Program Repair](https://mlsanity.com/paper/2106.08253/) (2021)
- [Deep Learning for Code Intelligence: Survey, Benchmark and Toolkit](https://mlsanity.com/paper/2401.00288/) (2023)
- [Benchmarking and Explaining Large Language Model-based Code Generation: A Causality-Centric Approach](https://mlsanity.com/paper/2310.06680/) (2023)
- [Incorporating Domain Knowledge through Task Augmentation for Front-End JavaScript Code Generation](https://mlsanity.com/paper/2208.10091/) (2022)
- [Improving Tree-Structured Decoder Training for Code Generation via Mutual Learning](https://mlsanity.com/paper/2105.14796/) (2021)
- [Exploring Dynamic Selection of Branch Expansion Orders for Code Generation](https://mlsanity.com/paper/2106.00261/) (2021)
- [Learning to Parallelize in a Shared-Memory Environment with Transformers](https://mlsanity.com/paper/2204.12835/) (2022)
- [Automatic Code Generation using Pre-Trained Language Models](https://mlsanity.com/paper/2102.10535/) (2021)
- [T5QL: Taming language models for SQL generation](https://mlsanity.com/paper/2209.10254/) (2022)
- [CodePAD: Sequence-based Code Generation with Pushdown Automaton](https://mlsanity.com/paper/2211.00818/) (2022)
- [On the Limitations of Embedding Based Methods for Measuring Functional Correctness for Code Generation](https://mlsanity.com/paper/2405.01580/) (2024)
- [Embedding API Dependency Graph for Neural Code Generation](https://mlsanity.com/paper/2103.15361/) (2021)
- [Grammar-Based Code Representation: Is It a Worthy Pursuit for LLMs?](https://mlsanity.com/paper/2503.05507/) (2025)
- [Multi-Granularity Detector for Vulnerability Fixes](https://mlsanity.com/paper/2305.13884/) (2023)
- [Modularizing while Training: A New Paradigm for Modularizing DNN Models](https://mlsanity.com/paper/2306.09376/) (2023)
- [SapientML: Synthesizing Machine Learning Pipelines by Learning from Human-Written Solutions](https://mlsanity.com/paper/2202.10451/) (2022)
- [Transformer with Tree-order Encoding for Neural Program Generation](https://mlsanity.com/paper/2206.13354/) (2022)
- [NeuDep: Neural Binary Memory Dependence Analysis](https://mlsanity.com/paper/2210.02853/) (2022)
- [Neural Machine Translation for Code Generation](https://mlsanity.com/paper/2305.13504/) (2023)
- [Structured Code Representations Enable Data-Efficient Adaptation of Code Language Models](https://mlsanity.com/paper/2401.10716/) (2024)
- [A Globally Normalized Neural Model for Semantic Parsing](https://mlsanity.com/paper/2106.03376/) (2021)
- [Speculative Decoding for Verilog: Speed and Quality, All in One](https://mlsanity.com/paper/2503.14153/) (2025)
- [Auto-COP: Adaptation Generation in Context-Oriented Programming using Reinforcement Learning Options](https://mlsanity.com/paper/2103.06757/) (2021)
- [Combining Improvements for Exploiting Dependency Trees in Neural Semantic Parsing](https://mlsanity.com/paper/2112.13179/) (2021)
- [Antecedent Predictions Are More Important Than You Think: An Effective Method for Tree-Based Code Generation](https://mlsanity.com/paper/2208.09998/) (2022)
- [Judging Adam: Studying the Performance of Optimization Methods on ML4SE Tasks](https://mlsanity.com/paper/2303.03540/) (2023)

## Abstract (arXiv)

Code generation maps a program description to executable source code in a programming language. Existing approaches mainly rely on a recurrent neural network (RNN) as the decoder. However, we find that a program contains significantly more tokens than a natural language sentence, and thus it may be inappropriate for RNN to capture such a long sequence. In this paper, we propose a grammar-based structural convolutional neural network (CNN) for code generation. Our model generates a program by predicting the grammar rules of the programming language; we design several CNN modules, including the tree-based convolution and pre-order convolution, whose information is further aggregated by dedicated attentive pooling layers. Experimental results on the HearthStone benchmark dataset show that our CNN code generator significantly outperforms the previous state-of-the-art method by 5 percentage points; additional experiments on several semantic parsing tasks demonstrate the robustness of our model. We also conduct in-depth ablation test to better understand each component of our model.