NL2Bash: A Corpus and Semantic Parser for Natural Language Interface to the Linux Operating System
Not on the map. Why?
Abstract (arXiv)
We present new data and semantic parsing methods for the problem of mapping English sentences to Bash commands (NL2Bash). Our long-term goal is to enable any user to perform operations such as file manipulation, search, and application-specific scripting by simply stating their goals in English. We take a first step in this domain, by providing a new dataset of challenging but commonly used Bash commands and expert-written English descriptions, along with baseline methods to establish performance levels on this task.
Key facts. Cited by 97 papers in the map (data through 7 Oct 2026). First posted on arXiv 25 Feb 2018.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 13; 2022: 13; 2023: 23; 2024: 18; 2025: 15; 2026: 15.
97 papers.
Sort:
By area:
- 2023AgentBench: Evaluating LLMs as Agentscited by 1,211
- 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,134
- 2021Cross-Task Generalization via Natural Language Crowdsourcing Instructionscited by 696
- 2023InterCode: Standardizing and Benchmarking Interactive Coding with Execution Feedbackcited by 229
- 2023LEVER: Learning to Verify Language-to-Code Generation with Executioncited by 228
- 2024Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflowscited by 223
- 2024ArCHer: Training Language Model Agents via Hierarchical Multi-Turn RLcited by 212
- 2022DocPrompting: Generating Code by Retrieving the Docscited by 151
- 2023Compositional Exemplars for In-context Learningcited by 143
- 2023NL4Opt Competition: Formulating Optimization Problems Based on Their Natural Language Descriptionscited by 133
- 2022Natural Language to Code Translation with Executioncited by 122
- 2021Case-based Reasoning for Natural Language Queries over Knowledge Basescited by 107
38 more
- 2025Large Language Models Often Know When They Are Being Evaluatedcited by 105
- 2024HiAgent: Hierarchical Working Memory Management for Solving Long-Horizon Agent Tasks with Large Language Modelcited by 101
- 2022Coder Reviewer Reranking for Code Generationcited by 83
- 2023Agent Lumos: Unified and Modular Training for Open-Source Language Agentscited by 72
- 2024In-context Learning with Retrieved Demonstrations for Language Models: A Surveycited by 69
- 2021Learning to Synthesize Programs as Interpretable and Generalizable Policiescited by 59
- 2024DA-Code: Agent Data Science Code Generation Benchmark for Large Language Modelscited by 57
- 2023ASSISTGUI: Task-Oriented Desktop Graphical User Interface Automationcited by 45
- 2025DABstep: Data Agent Benchmark for Multi-step Reasoningcited by 44
- 2026Endless Terminals: Scaling RL Environments for Terminal Agentscited by 43
- 2021SpreadsheetCoder: Formula Prediction from Semi-structured Contextcited by 41
- 2023Symbol-LLM: Towards Foundational Symbol-centric Interface For Large Language Modelscited by 38
- 2022Execution-based Evaluation for Data Science Code Generation Modelscited by 31
- 2023Learning Deep Semantics for Test Completioncited by 31
- 2023CodeFusion: A Pre-trained Diffusion Model for Code Generationcited by 31
- 2021SeaD: End-to-end Text-to-SQL Generation with Schema-aware Denoisingcited by 27
- 2023Natural Language Interfaces for Tabular Data Querying and Visualization: A Surveycited by 27
- 2026Tmax: A simple recipe for terminal agentscited by 24
- 2021DeepSTL -- From English Requirements to Signal Temporal Logiccited by 22
- 2023ByteSized32: A Corpus and Challenge Task for Generating Task-Specific World Models Expressed as Text Gamescited by 22
- 2021Investigating Math Word Problems using Pretrained Multilingual Language Modelscited by 21
- 2024Better than Your Teacher: LLM Agents that learn from Privileged AI Feedbackcited by 21
- 2022UniSAr: A Unified Structure-Aware Autoregressive Language Model for Text-to-SQLcited by 20
- 2023Hierarchical Programmatic Reinforcement Learning via Learning to Compose Programscited by 20
- 2023FIREBALL: A Dataset of Dungeons and Dragons Actual-Play with Structured Game State Informationcited by 18
- 2025SetupBench: Assessing Software Engineering Agents' Ability to Bootstrap Development Environmentscited by 18
- 2021Shellcode_IA32: A Dataset for Automatic Shellcode Generationcited by 16
- 2022Uni-Parser: Unified Semantic Parser for Question Answering on Knowledge Base and Databasecited by 16
- 2026SWE-Universe: Scale Real-World Verifiable Environments to Millionscited by 16
- 2026Toward Scalable Terminal Task Synthesis via Skill Graphscited by 16
- 2024On the Noise Robustness of In-Context Learning for Text Generationcited by 15
- 2025ExCyTIn-Bench: Evaluating LLM agents on Cyber Threat Investigationcited by 15
- 2022A Survey of Intent Classification and Slot-Filling Datasets for Task-Oriented Dialogcited by 14
- 2024Synthesizing Programmatic Reinforcement Learning Policies with Large Language Model Guided Searchcited by 13
- 2024PromptIntern: Saving Inference Costs by Internalizing Recurrent Prompt during Large Language Model Fine-tuningcited by 13
- 2021Multi-modal Program Inference: a Marriage of Pre-trainedLanguage Models and Component-based Synthesiscited by 12
- 2023Generating Data for Symbolic Language with Large Language Modelscited by 12
- 2024DocCGen: Document-based Controlled Code Generationcited by 11
50 most cited of 97 All 97 (JSON)
- 2026GRUET: Quantifying Uncertainty of Agentic Reasoning-and-Acting Processes21 Sep 2026
- 2026QuoteBench: How Matched Scores Can Hide Command-Path Failures13 Aug 2026
- 2026Look Before You Leap: Pre-Action Verification for LLM Agents9 Aug 2026
- 2026Meta-Task: Turning Terminal Task Synthesis into a Terminal Task for Scalable Agent Training30 Jul 2026
- 2026Gradient-free Task-Conditioned Retrieval for On-Device In-Context Learning30 Jul 2026
- 2026On-Policy Distillation for LLM Safety: A Routing Approach to Template-Robust Realignment29 Jul 2026
- 2026SETA: Scaling Environments for Terminal Agents12 Jul 2026
- 2026A Few Teacher Steps Go a Long Way: Cost-Efficient On-Policy Data Augmentation for Agent Post-Training6 Jul 2026
- 2026Tmax: A simple recipe for terminal agents22 Jun 2026
- 2026Efficient and Sound Probabilistic Verification for AI Agents18 Jun 2026
- 2026EvalDetectBench: A Benchmark for Measuring Evaluation Awareness in Frontier Language Models8 Jun 2026
- 2026TerminalWorld: Benchmarking Agents on Real-World Terminal Tasks21 May 2026
38 more
- 2026Toward Scalable Terminal Task Synthesis via Skill Graphs28 Apr 2026
- 2026SWE-Universe: Scale Real-World Verifiable Environments to Millions2 Feb 2026
- 2026Endless Terminals: Scaling RL Environments for Terminal Agents23 Jan 2026
- 2025SLMFix: Leveraging Small Language Models for Domain Specific Language Error Fixing with Reinforcement Learning24 Nov 2025
- 2025Prompt Engineering Techniques for Context-dependent Text-to-SQL in Arabic16 Nov 2025
- 2025Generative Caching for Structurally Similar Prompts and Responses14 Nov 2025
- 2025Learning "Partner-Aware" Collaborators in Multi-Party Collaboration26 Oct 2025
- 2025The Command Line GUIde: Graphical Interfaces from Man Pages via AI1 Oct 2025
- 2025Multilingual Text-to-SQL: Benchmarking the Limits of Language Models with Collaborative Language Agents29 Sep 2025
- 2025ExCyTIn-Bench: Evaluating LLM agents on Cyber Threat Investigation14 Jul 2025
- 2025SetupBench: Assessing Software Engineering Agents' Ability to Bootstrap Development Environments11 Jul 2025
- 2025DABstep: Data Agent Benchmark for Multi-step Reasoning30 Jun 2025
- 2025Selecting Demonstrations for Many-Shot In-Context Learning via Gradient Matching5 Jun 2025
- 2025Large Language Models Often Know When They Are Being Evaluated28 May 2025
- 2025ShIOEnv: A Command Evaluation Environment for Grammar-Constrained Synthesis and Execution Behavior Modeling23 May 2025
- 2025Benchmarking AI Models in Software Engineering: A Review, Search Tool, and Unified Approach for Elevating Benchmark Quality7 Mar 2025
- 2025Mapping the Landscape of Generative AI in Network Monitoring and Management12 Feb 2025
- 2025LLM-Supported Natural Language to Bash Translation7 Feb 2025
- 2024Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflows12 Nov 2024
- 2024DA-Code: Agent Data Science Code Generation Benchmark for Large Language Models9 Oct 2024
- 2024Better than Your Teacher: LLM Agents that learn from Privileged AI Feedback7 Oct 2024
- 2024Contextualized Data-Wrangling Code Generation in Computational Notebooks20 Sep 2024
- 2024ScriptSmith: A Unified LLM Framework for Enhancing IT Operations via Automated Bash Script Generation, Assessment, and Refinement12 Sep 2024
- 2024RACONTEUR: A Knowledgeable, Insightful, and Portable LLM-Powered Shell Command Explainer3 Sep 2024
- 2024HiAgent: Hierarchical Working Memory Management for Solving Long-Horizon Agent Tasks with Large Language Model18 Aug 2024
- 2024PromptIntern: Saving Inference Costs by Internalizing Recurrent Prompt during Large Language Model Fine-tuning2 Jul 2024
- 2024DocCGen: Document-based Controlled Code Generation17 Jun 2024
- 2024Unraveling the Mechanics of Learning-Based Demonstration Selection for In-Context Learning14 Jun 2024
- 2024Source Code Foundation Models are Transferable Binary Analysis Knowledge Bases30 May 2024
- 2024On the Noise Robustness of In-Context Learning for Text Generation27 May 2024
- 2024Synthesizing Programmatic Reinforcement Learning Policies with Large Language Model Guided Search26 May 2024
- 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments11 Apr 2024
- 2024ArCHer: Training Language Model Agents via Hierarchical Multi-Turn RL29 Feb 2024
- 2024Robust Synthetic Data-Driven Detection of Living-Off-the-Land Reverse Shells28 Feb 2024
- 2024NICE: To Optimize In-Context Examples or Not?9 Feb 2024
- 2024In-context Learning with Retrieved Demonstrations for Language Models: A Survey21 Jan 2024
- 2023ASSISTGUI: Task-Oriented Desktop Graphical User Interface Automation20 Dec 2023
- 2023Dynamic Retrieval-Augmented Generation14 Dec 2023
50 newest of 97 All 97 (JSON)
- 2023AgentBench: Evaluating LLMs as Agentscited by 1,211
- 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,134
- 2024ArCHer: Training Language Model Agents via Hierarchical Multi-Turn RLcited by 212
- 2023Agent Lumos: Unified and Modular Training for Open-Source Language Agentscited by 72
- 2024DA-Code: Agent Data Science Code Generation Benchmark for Large Language Modelscited by 57
- 2026Tmax: A simple recipe for terminal agentscited by 24
- 2024Better than Your Teacher: LLM Agents that learn from Privileged AI Feedbackcited by 21
- 2025SetupBench: Assessing Software Engineering Agents' Ability to Bootstrap Development Environmentscited by 18
- 2025ExCyTIn-Bench: Evaluating LLM agents on Cyber Threat Investigationcited by 15
- 2025Multilingual Text-to-SQL: Benchmarking the Limits of Language Models with Collaborative Language Agentscited by 7
- 2026SETA: Scaling Environments for Terminal Agentscited by 5
- 2025Learning "Partner-Aware" Collaborators in Multi-Party Collaborationcited by 2
12 most cited of 15 All 15 (JSON)
- 2026Look Before You Leap: Pre-Action Verification for LLM Agents9 Aug 2026
- 2026Meta-Task: Turning Terminal Task Synthesis into a Terminal Task for Scalable Agent Training30 Jul 2026
- 2026SETA: Scaling Environments for Terminal Agents12 Jul 2026
- 2026A Few Teacher Steps Go a Long Way: Cost-Efficient On-Policy Data Augmentation for Agent Post-Training6 Jul 2026
- 2026Tmax: A simple recipe for terminal agents22 Jun 2026
- 2025Learning "Partner-Aware" Collaborators in Multi-Party Collaboration26 Oct 2025
- 2025Multilingual Text-to-SQL: Benchmarking the Limits of Language Models with Collaborative Language Agents29 Sep 2025
- 2025ExCyTIn-Bench: Evaluating LLM agents on Cyber Threat Investigation14 Jul 2025
- 2025SetupBench: Assessing Software Engineering Agents' Ability to Bootstrap Development Environments11 Jul 2025
- 2024DA-Code: Agent Data Science Code Generation Benchmark for Large Language Models9 Oct 2024
- 2024Better than Your Teacher: LLM Agents that learn from Privileged AI Feedback7 Oct 2024
- 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments11 Apr 2024
12 newest of 15 All 15 (JSON)
- 2023LEVER: Learning to Verify Language-to-Code Generation with Executioncited by 228
- 2024HiAgent: Hierarchical Working Memory Management for Solving Long-Horizon Agent Tasks with Large Language Modelcited by 101
- 2025DABstep: Data Agent Benchmark for Multi-step Reasoningcited by 44
- 2026GRUET: Quantifying Uncertainty of Agentic Reasoning-and-Acting Processescited by 0
- 2026GRUET: Quantifying Uncertainty of Agentic Reasoning-and-Acting Processes21 Sep 2026
- 2025DABstep: Data Agent Benchmark for Multi-step Reasoning30 Jun 2025
- 2024HiAgent: Hierarchical Working Memory Management for Solving Long-Horizon Agent Tasks with Large Language Model18 Aug 2024
- 2023LEVER: Learning to Verify Language-to-Code Generation with Execution16 Feb 2023
- 2021Cross-Task Generalization via Natural Language Crowdsourcing Instructionscited by 696
- 2023InterCode: Standardizing and Benchmarking Interactive Coding with Execution Feedbackcited by 229
- 2024Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflowscited by 223
- 2022DocPrompting: Generating Code by Retrieving the Docscited by 151
- 2023Compositional Exemplars for In-context Learningcited by 143
- 2023NL4Opt Competition: Formulating Optimization Problems Based on Their Natural Language Descriptionscited by 133
- 2022Natural Language to Code Translation with Executioncited by 122
- 2021Case-based Reasoning for Natural Language Queries over Knowledge Basescited by 107
- 2025Large Language Models Often Know When They Are Being Evaluatedcited by 105
- 2022Coder Reviewer Reranking for Code Generationcited by 83
- 2024In-context Learning with Retrieved Demonstrations for Language Models: A Surveycited by 69
- 2021Learning to Synthesize Programs as Interpretable and Generalizable Policiescited by 59
12 most cited of 72 All 72 (JSON)
- 2026QuoteBench: How Matched Scores Can Hide Command-Path Failures13 Aug 2026
- 2026Gradient-free Task-Conditioned Retrieval for On-Device In-Context Learning30 Jul 2026
- 2026Efficient and Sound Probabilistic Verification for AI Agents18 Jun 2026
- 2026EvalDetectBench: A Benchmark for Measuring Evaluation Awareness in Frontier Language Models8 Jun 2026
- 2026TerminalWorld: Benchmarking Agents on Real-World Terminal Tasks21 May 2026
- 2026Toward Scalable Terminal Task Synthesis via Skill Graphs28 Apr 2026
- 2026SWE-Universe: Scale Real-World Verifiable Environments to Millions2 Feb 2026
- 2025SLMFix: Leveraging Small Language Models for Domain Specific Language Error Fixing with Reinforcement Learning24 Nov 2025
- 2025Prompt Engineering Techniques for Context-dependent Text-to-SQL in Arabic16 Nov 2025
- 2025Generative Caching for Structurally Similar Prompts and Responses14 Nov 2025
- 2025The Command Line GUIde: Graphical Interfaces from Man Pages via AI1 Oct 2025
- 2025Selecting Demonstrations for Many-Shot In-Context Learning via Gradient Matching5 Jun 2025
12 newest of 72 All 72 (JSON)