# Can LLMs Handle WebShell Detection? Overcoming Detection Challenges with Behavioral Function-Aware Framework

arXiv: https://arxiv.org/abs/2504.13811 · HTML: https://mlsanity.com/paper/2504.13811/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 45 references that are in the map.
- First posted on arXiv 14 Apr 2025.

## Builds on (most cited first, 23 of 23)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?](https://mlsanity.com/paper/2202.12837/) (2022)
- [Extending Context Window of Large Language Models via Positional Interpolation](https://mlsanity.com/paper/2306.15595/) (2023)
- [LongCoder: A Long-Range Pre-trained Language Model for Code Completion](https://mlsanity.com/paper/2306.14893/) (2023)
- [Mitigating Label Biases for In-context Learning](https://mlsanity.com/paper/2305.19148/) (2023)
- [LLM4Vuln: A Unified Evaluation Framework for Decoupling and Enhancing LLMs' Vulnerability Reasoning](https://mlsanity.com/paper/2401.16185/) (2024)
- [Large Language Models for Code Analysis: Do LLMs Really Do Their Job?](https://mlsanity.com/paper/2310.12357/) (2023)
- [Chain-of-Thought Prompting of Large Language Models for Discovering and Fixing Software Vulnerabilities](https://mlsanity.com/paper/2402.17230/) (2024)
- [On the Reliability and Explainability of Language Models for Program Generation](https://mlsanity.com/paper/2302.09587/) (2023)
- [SparseCoder: Identifier-Aware Sparse Transformer for File-Level Code Summarization](https://mlsanity.com/paper/2401.14727/) (2024)
- [Read Before You Think: Mitigating LLM Comprehension Failures with Step-by-Step Reading](https://mlsanity.com/paper/2504.09402/) (2025)
- [Towards Informative Few-Shot Prompt with Maximum Information Gain for In-Context Learning](https://mlsanity.com/paper/2310.08923/) (2023)
- [Focused Large Language Models are Stable Many-Shot Learners](https://mlsanity.com/paper/2408.13987/) (2024)
- [STACC: Code Comment Classification using SentenceTransformers](https://mlsanity.com/paper/2302.13149/) (2023)
- [Can LLM Prompting Serve as a Proxy for Static Analysis in Vulnerability Detection](https://mlsanity.com/paper/2412.12039/) (2024)
- [Enhancing Input-Label Mapping in In-Context Learning with Contrastive Decoding](https://mlsanity.com/paper/2502.13738/) (2025)
- [A ML-LLM pairing for better code comment classification](https://mlsanity.com/paper/2310.10275/) (2023)
- [Large Language Models are Few-shot Generators: Proposing Hybrid Prompt Algorithm To Generate Webshell Escape Samples](https://mlsanity.com/paper/2402.07408/) (2024)
- [Text Classification: Neural Networks VS Machine Learning Models VS Pre-trained Models](https://mlsanity.com/paper/2412.21022/) (2024)
- [Poster: Long PHP webshell files detection based on sliding window attention](https://mlsanity.com/paper/2502.19257/) (2025)
- [LLM-augmented Preference Learning from Natural Language](https://mlsanity.com/paper/2310.08523/) (2023)

## Built on by (most cited first, 6 of 6)

- [ZeroTuning: Unlocking the Initial Token's Power to Enhance Large Language Models Without Training](https://mlsanity.com/paper/2505.11739/) (2025)
- [A Reward-driven Automated Webshell Malicious-code Generator for Red-teaming](https://mlsanity.com/paper/2505.24252/) (2025)
- [Beyond Detection: A Comprehensive Benchmark and Study on Representation Learning for Fine-Grained Webshell Family Classification](https://mlsanity.com/paper/2512.05288/) (2025)
- [Hierarchical Dual-Strategy Unlearning for Biomedical and Healthcare Intelligence Using Imperfect and Privacy-Sensitive Medical Data](https://mlsanity.com/paper/2511.19498/) (2025)
- [ShellForge: Adversarial Co-Evolution of Webshell Generation and Multi-View Detection for Robust Webshell Defense](https://mlsanity.com/paper/2601.22182/) (2026)
- [Second-Order Problem Solving for Recursive Self-Improvement in Formal Verification](https://mlsanity.com/paper/2610.05701/) (2026)

## Abstract (arXiv)

WebShell attacks - where adversaries implant malicious scripts on web servers - remain a persistent threat. Prior machine-learning and deep-learning detectors typically depend on task-specific supervision and can be brittle under data scarcity, rapid concept drift, and out-of-distribution (OOD) deployment. Large language models (LLMs) have recently shown strong code understanding capabilities, but their reliability for WebShell detection remains unclear. We address this gap by (i) systematically evaluating seven LLMs (including GPT-4, LLaMA-3.1-70B, and Qwen-2.5 variants) against representative sequence- and graph-based baselines on 26.59K PHP scripts, and (ii) proposing Behavioral Function-Aware Detection (BFAD), a behavior-centric framework that adapts LLM inference to WebShell-specific execution patterns. BFAD anchors analysis on security-sensitive PHP functions via a Critical Function Filter, constructs compact LLM inputs with Context-Aware Code Extraction, and selects in-context demonstrations using Weighted Behavioral Function Profiling, which ranks examples by a behavior-weighted, function-level similarity. Empirically, we observe a consistent precision-recall asymmetry: larger LLMs often achieve high precision but miss attacks (lower recall), while smaller models exhibit the opposite tendency; moreover, off-the-shelf LLM prompting underperforms established detectors. BFAD substantially improves all evaluated LLMs, boosting F1 by 13.82% on average; notably, GPT-4, LLaMA-3.1-70B, and Qwen-2.5-Coder-14B exceed prior SOTA benchmarks, while Qwen-2.5-Coder-3B becomes competitive with traditional methods. Overall, our results clarify when LLMs succeed or fail on WebShell detection, provide a practical recipe, and highlight future directions for making LLM-based detection more reliable.