# Retrieval Augmented Generation Integrated Large Language Models in Smart Contract Vulnerability Detection

arXiv: https://arxiv.org/abs/2407.14838 · HTML: https://mlsanity.com/paper/2407.14838/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Privacy and security of RAG, one of 15 directions in Retrieval-augmented generation.
- Builds on 5 of its 19 references that are in the map.
- First posted on arXiv 20 Jul 2024.

## Placed in

- Retrieval-augmented generation / [Privacy and security of RAG](https://mlsanity.com/rag/privacy-and-security-of-rag/)

## Builds on (most cited first, 5 of 5)

- [Slither: A Static Analysis Framework For Smart Contracts](https://mlsanity.com/paper/1908.09878/) (2019)
- [Do you still need a manual smart contract audit?](https://mlsanity.com/paper/2306.12338/) (2023)
- [SoK: Decentralized Finance (DeFi) Attacks](https://mlsanity.com/paper/2208.13035/) (2022)
- [A Survey on Ethereum Systems Security: Vulnerabilities, Attacks and Defenses](https://mlsanity.com/paper/1908.04507/) (2019)
- [Consensus in the Age of Blockchains](https://mlsanity.com/paper/1711.03936/) (2017)

## Built on by (most cited first, 4 of 4)

- [SmartLLM: Smart Contract Auditing using Custom Generative AI](https://mlsanity.com/paper/2502.13167/) (2025)
- [A Knowledge Graph and a Tripartite Evaluation Framework Make Retrieval-Augmented Generation Scalable and Transparent](https://mlsanity.com/paper/2509.19209/) (2025)
- [ParaVul: A Parallel Large Language Model and Retrieval-Augmented Framework for Smart Contract Vulnerability Detection](https://mlsanity.com/paper/2510.17919/) (2025)
- [Three Heads Are Better Than One: A Multi-perspective Reasoning Framework for Enhanced Vulnerability Detection](https://mlsanity.com/paper/2605.18153/) (2026)

## Abstract (arXiv)

The rapid growth of Decentralized Finance (DeFi) has been accompanied by substantial financial losses due to smart contract vulnerabilities, underscoring the critical need for effective security auditing. With attacks becoming more frequent, the necessity and demand for auditing services has escalated. This especially creates a financial burden for independent developers and small businesses, who often have limited available funding for these services. Our study builds upon existing frameworks by integrating Retrieval-Augmented Generation (RAG) with large language models (LLMs), specifically employing GPT-4-1106 for its 128k token context window. We construct a vector store of 830 known vulnerable contracts, leveraging Pinecone for vector storage, OpenAI's text-embedding-ada-002 for embeddings, and LangChain to construct the RAG-LLM pipeline. Prompts were designed to provide a binary answer for vulnerability detection. We first test 52 smart contracts 40 times each against a provided vulnerability type, verifying the replicability and consistency of the RAG-LLM. Encouraging results were observed, with a 62.7% success rate in guided detection of vulnerabilities. Second, we challenge the model under a "blind" audit setup, without the vulnerability type provided in the prompt, wherein 219 contracts undergo 40 tests each. This setup evaluates the general vulnerability detection capabilities without hinted context assistance. Under these conditions, a 60.71% success rate was observed. While the results are promising, we still emphasize the need for human auditing at this time. We provide this study as a proof of concept for a cost-effective smart contract auditing process, moving towards democratic access to security.