# NeurIPS 2020 NLC2CMD Competition: Translating Natural Language to Bash Commands

arXiv: https://arxiv.org/abs/2103.02523 · HTML: https://mlsanity.com/paper/2103.02523/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 42 references that are in the map.
- First posted on arXiv 3 Mar 2021.

## Builds on (most cited first, 22 of 22)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020)
- [Energy and Policy Considerations for Deep Learning in NLP](https://mlsanity.com/paper/1906.02243/) (2019)
- [Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task](https://mlsanity.com/paper/1809.08887/) (2018)
- [Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learning](https://mlsanity.com/paper/1709.00103/) (2017)
- [CodeSearchNet Challenge: Evaluating the State of Semantic Code Search](https://mlsanity.com/paper/1909.09436/) (2019)
- [StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networks](https://mlsanity.com/paper/1612.03242/) (2016)
- [Unsupervised Translation of Programming Languages](https://mlsanity.com/paper/2006.03511/) (2020)
- [Towards the Systematic Reporting of the Energy and Carbon Footprints of Machine Learning](https://mlsanity.com/paper/2002.05651/) (2020)
- [A Survey of Machine Learning for Big Code and Naturalness](https://mlsanity.com/paper/1709.06182/) (2017)
- [Survey of the State of the Art in Natural Language Generation: Core tasks, applications and evaluation](https://mlsanity.com/paper/1703.09902/) (2017)
- [Semantic Evaluation for Text-to-SQL with Distilled Test Suites](https://mlsanity.com/paper/2010.02840/) (2020)
- [NL2Bash: A Corpus and Semantic Parser for Natural Language Interface to the Linux Operating System](https://mlsanity.com/paper/1802.08979/) (2018)
- [Tree-to-tree Neural Networks for Program Translation](https://mlsanity.com/paper/1802.03691/) (2018)
- [Does BLEU Score Work for Code Migration?](https://mlsanity.com/paper/1906.04903/) (2019)
- [Energy Usage Reports: Environmental awareness as part of algorithmic accountability](https://mlsanity.com/paper/1911.08354/) (2019)
- [From Language to Programs: Bridging Reinforcement Learning and Maximum Marginal Likelihood](https://mlsanity.com/paper/1704.07926/) (2017)
- [EvalAI: Towards Better Evaluation Systems for AI Agents](https://mlsanity.com/paper/1902.03570/) (2019)
- [Why Self-Attention? A Targeted Evaluation of Neural Machine Translation Architectures](https://mlsanity.com/paper/1808.08946/) (2018)
- [Photon: A Robust Cross-Domain Text-to-SQL System](https://mlsanity.com/paper/2007.15280/) (2020)
- [Project CLAI: Instrumenting the Command Line as a New Environment for AI Agents](https://mlsanity.com/paper/2002.00762/) (2020)

## Built on by (most cited first, 7 of 7)

- [InterCode: Standardizing and Benchmarking Interactive Coding with Execution Feedback](https://mlsanity.com/paper/2306.14898/) (2023)
- [NL2CMD: An Updated Workflow for Natural Language to Bash Commands Translation](https://mlsanity.com/paper/2302.07845/) (2023)
- [LLM-Supported Natural Language to Bash Translation](https://mlsanity.com/paper/2502.06858/) (2025)
- [Dynamic Retrieval-Augmented Generation](https://mlsanity.com/paper/2312.08976/) (2023)
- [QuoteBench: How Matched Scores Can Hide Command-Path Failures](https://mlsanity.com/paper/2608.13547/) (2026)
- [Structural Transfer Learning in NL-to-Bash Semantic Parsers](https://mlsanity.com/paper/2307.16795/) (2023)
- [ShIOEnv: A Command Evaluation Environment for Grammar-Constrained Synthesis and Execution Behavior Modeling](https://mlsanity.com/paper/2505.18374/) (2025)

## Abstract (arXiv)

The NLC2CMD Competition hosted at NeurIPS 2020 aimed to bring the power of natural language processing to the command line. Participants were tasked with building models that can transform descriptions of command line tasks in English to their Bash syntax. This is a report on the competition with details of the task, metrics, data, attempted solutions, and lessons learned.