# The impact of lexical and grammatical processing on generating code from natural language

arXiv: https://arxiv.org/abs/2202.13972 · HTML: https://mlsanity.com/paper/2202.13972/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 19 references that are in the map.
- First posted on arXiv 28 Feb 2022.

## Builds on (most cited first, 13 of 13)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Program Synthesis with Large Language Models](https://mlsanity.com/paper/2108.07732/) (2021)
- [Effective Approaches to Attention-based Neural Machine Translation](https://mlsanity.com/paper/1508.04025/) (2015)
- [Measuring Coding Challenge Competence With APPS](https://mlsanity.com/paper/2105.09938/) (2021)
- [Pointer Networks](https://mlsanity.com/paper/1506.03134/) (2015)
- [Language to Logical Form with Neural Attention](https://mlsanity.com/paper/1601.01280/) (2016)
- [Abstract Syntax Networks for Code Generation and Semantic Parsing](https://mlsanity.com/paper/1704.07535/) (2017)
- [TRANX: A Transition-based Neural Abstract Syntax Parser for Semantic Parsing and Code Generation](https://mlsanity.com/paper/1810.02720/) (2018)
- [Reading StackOverflow Encourages Cheating: Adding Question Text Improves Extractive Code Generation](https://mlsanity.com/paper/2106.04447/) (2021)
- [Code Generation from Natural Language with Less Prior and More Monolingual Data](https://mlsanity.com/paper/2101.00259/) (2021)

## Built on by (most cited first, 6 of 6)

- [Retrieval-Augmented Generation for AI-Generated Content: A Survey](https://mlsanity.com/paper/2402.19473/) (2024)
- [DocPrompting: Generating Code by Retrieving the Docs](https://mlsanity.com/paper/2207.05987/) (2022)
- [Automatic Readability Assessment for Closely Related Languages](https://mlsanity.com/paper/2305.13478/) (2023)
- [InfeRE: Step-by-Step Regex Generation via Chain of Inference](https://mlsanity.com/paper/2308.04041/) (2023)
- [CodeInsight: A Curated Dataset of Practical Coding Solutions from Stack Overflow](https://mlsanity.com/paper/2409.16819/) (2024)
- [RETROcode: Leveraging a Code Database for Improved Natural Language to Code Generation](https://mlsanity.com/paper/2504.05759/) (2025)

## Abstract (arXiv)

Considering the seq2seq architecture of TranX for natural language to code translation, we identify four key components of importance: grammatical constraints, lexical preprocessing, input representations, and copy mechanisms. To study the impact of these components, we use a state-of-the-art architecture that relies on BERT encoder and a grammar-based decoder for which a formalization is provided. The paper highlights the importance of the lexical substitution component in the current natural language to code systems.