mlsanityData to 7 Oct 2026

NeurIPS 2020 NLC2CMD Competition: Translating Natural Language to Bash Commands

arXiv 2103.02523 ↗First posted 3 Mar 2021cs.CLcited by 7

Not on the map. Why?

Abstract (arXiv)

The NLC2CMD Competition hosted at NeurIPS 2020 aimed to bring the power of natural language processing to the command line. Participants were tasked with building models that can transform descriptions of command line tasks in English to their Bash syntax. This is a report on the competition with details of the task, metrics, data, attempted solutions, and lessons learned.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 22 of its 42 references that are in the map. First posted on arXiv 3 Mar 2021.

Builds on

22 of 42 references.

References · 22

  1. 2017Attention Is All You Needcited by 59,932
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2020Extracting Training Data from Large Language Modelscited by 2,144
  4. 2019Energy and Policy Considerations for Deep Learning in NLPcited by 1,242
  5. 2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,120
  6. 2017Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learningcited by 679
  7. 2019CodeSearchNet Challenge: Evaluating the State of Semantic Code Searchcited by 638
  8. 2016StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networkscited by 575
  9. 2020Unsupervised Translation of Programming Languagescited by 262
  10. 2020Towards the Systematic Reporting of the Energy and Carbon Footprints of Machine Learningcited by 246
  11. 2017A Survey of Machine Learning for Big Code and Naturalnesscited by 220
  12. 2017Survey of the State of the Art in Natural Language Generation: Core tasks, applications and evaluationcited by 206
10 more
  1. 2020Semantic Evaluation for Text-to-SQL with Distilled Test Suitescited by 123
  2. 2018NL2Bash: A Corpus and Semantic Parser for Natural Language Interface to the Linux Operating Systemcited by 97
  3. 2018Tree-to-tree Neural Networks for Program Translationcited by 93
  4. 2019Does BLEU Score Work for Code Migration?cited by 49
  5. 2019Energy Usage Reports: Environmental awareness as part of algorithmic accountabilitycited by 43
  6. 2017From Language to Programs: Bridging Reinforcement Learning and Maximum Marginal Likelihoodcited by 41
  7. 2019EvalAI: Towards Better Evaluation Systems for AI Agentscited by 34
  8. 2018Why Self-Attention? A Targeted Evaluation of Neural Machine Translation Architecturescited by 30
  9. 2020Photon: A Robust Cross-Domain Text-to-SQL Systemcited by 26
  10. 2020Project CLAI: Instrumenting the Command Line as a New Environment for AI Agentscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 4; 2024: 0; 2025: 2; 2026: 1.

7 papers.

  1. 2023InterCode: Standardizing and Benchmarking Interactive Coding with Execution Feedbackcited by 229
  2. 2023NL2CMD: An Updated Workflow for Natural Language to Bash Commands Translationcited by 3
  3. 2025LLM-Supported Natural Language to Bash Translationcited by 2
  4. 2023Dynamic Retrieval-Augmented Generationcited by 1
  5. 2023Structural Transfer Learning in NL-to-Bash Semantic Parserscited by 0
  6. 2025ShIOEnv: A Command Evaluation Environment for Grammar-Constrained Synthesis and Execution Behavior Modelingcited by 0
  7. 2026QuoteBench: How Matched Scores Can Hide Command-Path Failurescited by 0