# ParaVul: A Parallel Large Language Model and Retrieval-Augmented Framework for Smart Contract Vulnerability Detection arXiv: https://arxiv.org/abs/2510.17919 ยท HTML: https://mlsanity.com/paper/2510.17919/ ## Key facts - Cited by 1 papers in the map (data through 14 Oct 2026). - Placed in Privacy and security of RAG, one of 15 directions in Retrieval-augmented generation. - Builds on 28 of its 53 references that are in the map. - First posted on arXiv 20 Oct 2025. ## Placed in - Retrieval-augmented generation / [Privacy and security of RAG](https://mlsanity.com/rag/privacy-and-security-of-rag/) ## Builds on (most cited first, 28 of 28) - [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017) - [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018) - [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021) - [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023) - [Similarity of Neural Network Representations Revisited](https://mlsanity.com/paper/1905.00414/) (2019) - [SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compression](https://mlsanity.com/paper/2306.03078/) (2023) - [QA-LoRA: Quantization-Aware Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2309.14717/) (2023) - [Dense Text Retrieval based on Pretrained Language Models: A Survey](https://mlsanity.com/paper/2211.14876/) (2022) - [Sparse Low-rank Adaptation of Pre-trained Language Models](https://mlsanity.com/paper/2311.11696/) (2023) - [Slither: A Static Analysis Framework For Smart Contracts](https://mlsanity.com/paper/1908.09878/) (2019) - [GPTScan: Detecting Logic Vulnerabilities in Smart Contracts by Combining GPT with Program Analysis](https://mlsanity.com/paper/2308.03314/) (2023) - [Securify: Practical Security Analysis of Smart Contracts](https://mlsanity.com/paper/1806.01143/) (2018) - [LongEmbed: Extending Embedding Models for Long Context Retrieval](https://mlsanity.com/paper/2404.12096/) (2024) - [SLTrain: a sparse plus low-rank approach for parameter and memory efficient pretraining](https://mlsanity.com/paper/2406.02214/) (2024) - [When ChatGPT Meets Smart Contract Vulnerability Detection: How Far Are We?](https://mlsanity.com/paper/2309.05520/) (2023) - [Cost-Effective Hyperparameter Optimization for Large Language Model Generation Inference](https://mlsanity.com/paper/2303.04673/) (2023) - [PropertyGPT: LLM-driven Formal Verification of Smart Contracts through Retrieval-Augmented Property Generation](https://mlsanity.com/paper/2405.02580/) (2024) - [Blockchain-empowered Federated Learning for Healthcare Metaverses: User-centric Incentive Mechanism with Optimal Data Freshness](https://mlsanity.com/paper/2307.15975/) (2023) - [Interaction-Aware Vulnerability Detection in Smart Contract Bytecodes](https://mlsanity.com/paper/2410.20712/) (2024) - [SoK: Security Analysis of Blockchain-based Cryptocurrency](https://mlsanity.com/paper/2503.22156/) (2025) - [MetaGater: Fast Learning of Conditional Channel Gated Networks via Federated Meta-Learning](https://mlsanity.com/paper/2011.12511/) (2020) - [ScrawlD: A Dataset of Real World Ethereum Smart Contracts Labelled with Vulnerabilities](https://mlsanity.com/paper/2202.11409/) (2022) - [Retrieval Augmented Generation Integrated Large Language Models in Smart Contract Vulnerability Detection](https://mlsanity.com/paper/2407.14838/) (2024) - [LoRS: Efficient Low-Rank Adaptation for Sparse Large Language Model](https://mlsanity.com/paper/2501.08582/) (2025) - [SmartLLM: Smart Contract Auditing using Custom Generative AI](https://mlsanity.com/paper/2502.13167/) (2025) - [Optimal AoI-based Block Propagation and Incentive Mechanism for Blockchain Networks in Web 3.0](https://mlsanity.com/paper/2403.12807/) (2024) - [Graph Attention Network-based Block Propagation with Optimal AoI and Reputation in Web 3.0](https://mlsanity.com/paper/2403.13237/) (2024) - [LLM-BSCVM: An LLM-Based Blockchain Smart Contract Vulnerability Management Framework](https://mlsanity.com/paper/2505.17416/) (2025) ## Built on by (most cited first, 1 of 1) - [Frequency-Aware Continual Learning for Smart Contract Vulnerability Detection with Large Language Models](https://mlsanity.com/paper/2608.19680/) (2026) ## Abstract (arXiv) Smart contracts play a significant role in automating blockchain services. Nevertheless, vulnerabilities in smart contracts pose serious threats to blockchain security. Currently, traditional detection methods primarily rely on static analysis and formal verification, which can result in high false-positive rates and poor scalability. Large Language Models (LLMs) have recently made significant progress in smart contract vulnerability detection. However, they still face challenges such as high inference costs and substantial computational overhead. In this paper, we propose ParaVul, a parallel LLM and retrieval-augmented framework to improve the reliability and accuracy of smart contract vulnerability detection. Specifically, we first develop Sparse Low-Rank Adaptation (SLoRA) for LLM fine-tuning. SLoRA introduces sparsification by incorporating a sparse matrix into quantized LoRA-based LLMs, thereby reducing computational overhead and resource requirements while enhancing their ability to understand vulnerability-related issues. We then construct a vulnerability contract dataset and develop a hybrid Retrieval-Augmented Generation (RAG) system that integrates dense retrieval with Best Matching 25 (BM25), assisting in verifying the results generated by the LLM. Furthermore, we propose a meta-learning model to fuse the outputs of the RAG system and the LLM, thereby generating the final detection results. After completing vulnerability detection, we design chain-of-thought prompts to guide LLMs to generate comprehensive vulnerability detection reports. Simulation results demonstrate the superiority of ParaVul, especially in terms of F1 scores, achieving 0.9398 for single-label detection and 0.9330 for multi-label detection.