mlsanityData to 14 Oct 2026

Detecting Malicious Source Code in PyPI Packages with LLMs: Does RAG Come in Handy?

arXiv 2504.13769 ↗First posted 18 Apr 2025cs.SEcited by 3

On the map: Retrieval-augmented generation → Privacy and security of RAG

Abstract (arXiv)

Malicious software packages in open-source ecosystems, such as PyPI, pose growing security risks. Unlike traditional vulnerabilities, these packages are intentionally designed to deceive users, making detection challenging due to evolving attack methods and the lack of structured datasets. In this work, we empirically evaluate the effectiveness of Large Language Models (LLMs), Retrieval-Augmented Generation (RAG), and few-shot learning for detecting malicious source code. We fine-tune LLMs on curated datasets and integrate YARA rules, GitHub Security Advisories, and malicious code snippets with the aim of enhancing classification accuracy. We came across a counterintuitive outcome: While RAG is expected to boost up the prediction performance, it fails in the performed evaluation, obtaining a mediocre accuracy. In contrast, few-shot learning is more effective as it significantly improves the detection of malicious code, achieving 97% accuracy and 95% balanced accuracy, outperforming traditional RAG approaches. Thus, future work should expand structured knowledge bases, refine retrieval models, and explore hybrid AI-driven cybersecurity solutions.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Privacy and security of RAG, one of 15 directions in Retrieval-augmented generation. Builds on 11 of its 35 references that are in the map. First posted on arXiv 18 Apr 2025.

Builds on

11 of 35 references.

References · 11

  1. 2023Large Language Models for Software Engineering: Survey and Open Problemscited by 181
  2. 2023Self-planning Code Generation with Large Language Modelscited by 124
  3. 2024A New Era in LLM Security: Exploring Security Concerns in Real-World LLM-based Systemscited by 77
  4. 2024Vul-RAG: Enhancing LLM-based Vulnerability Detection via Knowledge-level RAGcited by 68
  5. 2020Backstabber's Knife Collection: A Review of Open Source Software Supply Chain Attackscited by 61
  6. 2024PenHeal: A Two-Stage LLM Framework for Automated Pentesting and Optimal Remediationcited by 30
  7. 2023An Empirical Study of Malicious Code In PyPI Ecosystemcited by 19
  8. 2022A Comparison of Static, Dynamic, and Hybrid Analysis for Malware Detectioncited by 17
  9. 2024Do Machines and Humans Focus on Similar Code? Exploring Explainability of Large Language Models in Code Summarizationcited by 16
  10. 2024LProtector: An LLM-driven Vulnerability Detection Systemcited by 8
  11. 2021Development of recommendation systems for software engineering: the CROSSMINER experiencecited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2026Many Hands Make Light Work: An LLM-based Multi-Agent System for Detecting Malicious PyPI Packagescited by 1
  2. 2025One Detector Fits All: Robust and Adaptive Detection of Malicious Packages from PyPI to Enterprisescited by 0
  3. 2026MOLOT System Card: Malicious Operational Logic Observation Transformercited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026An Empirical Evaluation of LLM-Based Approaches for Code Vulnerability Detection: RAG, SFT, and Dual-Agent Systems
  2. 2026Revisiting Vul-RAG: Reproducibility and Replicability of RAG-based Vulnerability Detection with Open-Weight Models
  3. 2025SOSecure: Safer Code Generation with RAG and StackOverflow Discussions
  4. 2026PYPILINE: Malicious PyPI Package Detection via Suspicious API Knowledge and Agent Workflow
  5. 2024Evaluating and Improving the Robustness of Security Attack Detectors Generated by LLMs
  6. 2025A Systematic Study of Code Obfuscation Against LLM-based Vulnerability Detection
  7. 2025"Digital Camouflage": The LLVM Challenge in LLM-Based Malware Detection
  8. 2023TrojanedCM: A Repository of Trojaned Large Language Models of Code