# Decoupled Smart Contract Audits: Lightweight LLM Framework via Distillation and Aggregation

arXiv: https://arxiv.org/abs/2606.03128 · HTML: https://mlsanity.com/paper/2606.03128/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 30 of its 45 references that are in the map.
- First posted on arXiv 2 Jun 2026.

## Builds on (most cited first, 30 of 30)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Code Llama: Open Foundation Models for Code](https://mlsanity.com/paper/2308.12950/) (2023)
- [Qwen2.5-Coder Technical Report](https://mlsanity.com/paper/2409.12186/) (2024)
- [DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligence](https://mlsanity.com/paper/2401.14196/) (2024)
- [Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizes](https://mlsanity.com/paper/2305.02301/) (2023)
- [Chain-of-Verification Reduces Hallucination in Large Language Models](https://mlsanity.com/paper/2309.11495/) (2023)
- [A Rank Stabilization Scaling Factor for Fine-Tuning with LoRA](https://mlsanity.com/paper/2312.03732/) (2023)
- [CodeGemma: Open Code Models Based on Gemma](https://mlsanity.com/paper/2406.11409/) (2024)
- [RewardBench 2: Advancing Reward Model Evaluation](https://mlsanity.com/paper/2506.01937/) (2025)
- [GPTScan: Detecting Logic Vulnerabilities in Smart Contracts by Combining GPT with Program Analysis](https://mlsanity.com/paper/2308.03314/) (2023)
- [LLM4Vuln: A Unified Evaluation Framework for Decoupling and Enhancing LLMs' Vulnerability Reasoning](https://mlsanity.com/paper/2401.16185/) (2024)
- [Beyond Answers: Transferring Reasoning Capabilities to Smaller LLMs Using Multi-Teacher Knowledge Distillation](https://mlsanity.com/paper/2402.04616/) (2024)
- [Combining Fine-Tuning and LLM-based Agents for Intuitive Smart Contract Auditing with Justifications](https://mlsanity.com/paper/2403.16073/) (2024)
- [Do you still need a manual smart contract audit?](https://mlsanity.com/paper/2306.12338/) (2023)
- [LMUnit: Fine-grained Evaluation with Natural Language Unit Tests](https://mlsanity.com/paper/2412.13091/) (2024)
- [Atla Selene Mini: A General Purpose Evaluation Model](https://mlsanity.com/paper/2501.17195/) (2025)
- [LLM-SmartAudit: Advanced Smart Contract Vulnerability Detection](https://mlsanity.com/paper/2410.09381/) (2024)
- [AuditGPT: Auditing Smart Contracts with ChatGPT](https://mlsanity.com/paper/2404.04306/) (2024)
- [Decoupling Task-Solving and Output Formatting in LLM Generation](https://mlsanity.com/paper/2510.03595/) (2025)
- [Smart-LLaMA: Two-Stage Post-Training of Large Language Models for Smart Contract Vulnerability Detection and Explanation](https://mlsanity.com/paper/2411.06221/) (2024)
- [Assessing Small Language Models for Code Generation: An Empirical Study with Benchmarks](https://mlsanity.com/paper/2507.03160/) (2025)
- [Small Language Models for Efficient Agentic Tool Calling: Outperforming Large Models with Targeted Fine-tuning](https://mlsanity.com/paper/2512.15943/) (2025)
- [SmartLLMSentry: A Comprehensive LLM Based Smart Contract Vulnerability Detection Framework](https://mlsanity.com/paper/2411.19234/) (2024)
- [A Large-Scale Collection Of (Non-)Actionable Static Code Analysis Reports](https://mlsanity.com/paper/2511.10323/) (2025)
- [Large Language Model based Smart Contract Auditing with LLMBugScanner](https://mlsanity.com/paper/2512.02069/) (2025)

## Abstract (arXiv)

Smart contracts face critical security challenges that require thorough auditing in decentralized web services. While Large Language Models (LLMs) have shown promise in automated vulnerability detection, existing approaches lack severity evaluations with actionable remediation and demand unnecessarily massive computational overhead. In this study, we introduce an efficient end-to-end smart contract security audit framework utilizing lightweight, highly optimized open-source LLMs (0.6B-4B parameters). Our framework decouples comprehensive audit tasks into four interconnected components: vulnerability detection, explanation, severity classification, and remediation recommendation. To maintain high accuracy without massive parameters, we implement Rank-Stabilized Low-Rank Adapters (rsLoRA), knowledge distillation, and a custom Chain-of-Verification (CoVe) aggregation strategy to systematically screen and consolidate multiple draft responses from the model into a highly accurate audit report. Experimental results demonstrate that our lightweight pipeline consistently outperforms state-of-the-art open-source coder dense LLMs (7B to 34B parameters), achieving 98.25% accuracy in vulnerability detection and an alignment score of 0.4375 in generative explanation tasks. Furthermore, our extensive ablation studies empirically validate the superiority of our decoupled audit processes over unified prompting and uncover a novel severity centrality bias, establishing a critical benchmark for future research in LLM-assisted auditing.