# Assessing the Effectiveness of LLMs in Android Application Vulnerability Analysis

arXiv: https://arxiv.org/abs/2406.18894 · HTML: https://mlsanity.com/paper/2406.18894/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Jun 2024.

## Built on by (most cited first, 6 of 6)

- [LLMs in Software Security: A Survey of Vulnerability Detection Techniques and Insights](https://mlsanity.com/paper/2502.07049/) (2025)
- [From Vulnerabilities to Remediation: A Systematic Literature Review of LLMs in Code Security](https://mlsanity.com/paper/2412.15004/) (2024)
- [Vulnerability Detection: From Formal Verification to Large Language Models and Hybrid Approaches: A Comprehensive Overview](https://mlsanity.com/paper/2503.10784/) (2025)
- [One Bug, Hundreds Behind: LLMs for Large-Scale Bug Discovery](https://mlsanity.com/paper/2510.14036/) (2025)
- [CORTEX: Composite Overlay for Risk Tiering and Exposure in Operational AI Systems](https://mlsanity.com/paper/2508.19281/) (2025)
- [From User Interface to Agent Interface: Efficiency Optimization of UI Representations for LLM Agents](https://mlsanity.com/paper/2512.13438/) (2025)

## Abstract (arXiv)

The increasing frequency of attacks on Android applications coupled with the recent popularity of large language models (LLMs) necessitates a comprehensive understanding of the capabilities of the latter in identifying potential vulnerabilities, which is key to mitigate the overall risk. To this end, the work at hand compares the ability of nine state-of-the-art LLMs to detect Android code vulnerabilities listed in the latest Open Worldwide Application Security Project (OWASP) Mobile Top 10. Each LLM was evaluated against an open dataset of over 100 vulnerable code samples, including obfuscated ones, assessing each model's ability to identify key vulnerabilities. Our analysis reveals the strengths and weaknesses of each LLM, identifying important factors that contribute to their performance. Additionally, we offer insights into context augmentation with retrieval-augmented generation (RAG) for detecting Android code vulnerabilities, which in turn may propel secure application development. Finally, while the reported findings regarding code vulnerability analysis show promise, they also reveal significant discrepancies among the different LLMs.