# Shell Language Processing: Unix command parsing for Machine Learning

arXiv: https://arxiv.org/abs/2107.02438 · HTML: https://mlsanity.com/paper/2107.02438/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 1 of its 4 references that are in the map.
- First posted on arXiv 6 Jul 2021.

## Builds on (most cited first, 1 of 1)

- [NL2Bash: A Corpus and Semantic Parser for Natural Language Interface to the Linux Operating System](https://mlsanity.com/paper/1802.08979/) (2018)

## Built on by (most cited first, 3 of 3)

- [BashExplainer: Retrieval-Augmented Bash Code Comment Generation based on Fine-tuned CodeBERT](https://mlsanity.com/paper/2206.13325/) (2022)
- [Nebula: Self-Attention for Dynamic Malware Analysis](https://mlsanity.com/paper/2310.10664/) (2023)
- [Robust Synthetic Data-Driven Detection of Living-Off-the-Land Reverse Shells](https://mlsanity.com/paper/2402.18329/) (2024)

## Abstract (arXiv)

In this article, we present a Shell Language Preprocessing (SLP) library, which implements tokenization and encoding directed at parsing Unix and Linux shell commands. We describe the rationale behind the need for a new approach with specific examples of when conventional Natural Language Processing (NLP) pipelines fail. Furthermore, we evaluate our methodology on a security classification task against widely accepted information and communications technology (ICT) tokenization techniques and achieve significant improvement of an F1 score from 0.392 to 0.874.