# A Benchmark Comparison of Python Malware Detection Approaches

arXiv: https://arxiv.org/abs/2209.13288 · HTML: https://mlsanity.com/paper/2209.13288/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Sep 2022.

## Built on by (most cited first, 7 of 7)

- [We Have a Package for You! A Comprehensive Analysis of Package Hallucinations by Code Generating LLMs](https://mlsanity.com/paper/2406.10279/) (2024)
- [Leveraging Large Language Models to Detect npm Malicious Packages](https://mlsanity.com/paper/2403.12196/) (2024)
- [One Detector Fits All: Robust and Adaptive Detection of Malicious Packages from PyPI to Enterprises](https://mlsanity.com/paper/2512.04338/) (2025)
- [CHASE: LLM Agents for Dissecting Malicious PyPI Packages](https://mlsanity.com/paper/2601.06838/) (2026)
- [eDySec: A Deep Learning-based Explainable Dynamic Analysis Framework for Detecting Malicious Packages in PyPI Ecosystem](https://mlsanity.com/paper/2604.26219/) (2026)
- [LLM-Enhanced Hierarchical Heterogeneous Graph Representation Learning for Malicious Python Package Detection](https://mlsanity.com/paper/2607.03350/) (2026)
- [After the Party: Growth, Governance, and Security Scanning in the OpenClaw Agent Skill Ecosystem](https://mlsanity.com/paper/2609.17274/) (2026)

## Abstract (arXiv)

While attackers often distribute malware to victims via open-source, community-driven package repositories, these repositories do not currently run automated malware detection systems. In this work, we explore the security goals of the repository administrators and the requirements for deployments of such malware scanners via a case study of the Python ecosystem and PyPI repository, which includes interviews with administrators and maintainers. Further, we evaluate existing malware detection techniques for deployment in this setting by creating a benchmark dataset and comparing several existing tools, including the malware checks implemented in PyPI, Bandit4Mal, and OSSGadget's OSS Detect Backdoor. We find that repository administrators have exacting technical demands for such malware detection tools. Specifically, they consider a false positive rate of even 0.01% to be unacceptably high, given the large number of package releases that might trigger false alerts. Measured tools have false positive rates between 15% and 97%; increasing thresholds for detection rules to reduce this rate renders the true positive rate useless. In some cases, these checks emitted alerts more often for benign packages than malicious ones. However, we also find a successful socio-technical malware detection system: external security researchers also perform repository malware scans and report the results to repository administrators. These parties face different incentives and constraints on their time and tooling. We conclude with recommendations for improving detection capabilities and strengthening the collaboration between security researchers and software repository administrators.