# Combining Fine-Tuning and LLM-based Agents for Intuitive Smart Contract Auditing with Justifications

arXiv: https://arxiv.org/abs/2403.16073 · HTML: https://mlsanity.com/paper/2403.16073/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use.
- Builds on 51 of its 82 references that are in the map.
- First posted on arXiv 24 Mar 2024.

## Placed in

- LLM agents and tool use / [Agent threats, guardrails and governance](https://mlsanity.com/agents/agent-threats-guardrails-and-governance/)

## Builds on (most cited first, 30 of 51)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://mlsanity.com/paper/2101.00190/) (2021)
- [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021)
- [Mistral 7B](https://mlsanity.com/paper/2310.06825/) (2023)
- [A Survey of Large Language Models](https://mlsanity.com/paper/2303.18223/) (2023)
- [Code Llama: Open Foundation Models for Code](https://mlsanity.com/paper/2308.12950/) (2023)
- [Mixtral of Experts](https://mlsanity.com/paper/2401.04088/) (2024)
- [CodeBERT: A Pre-Trained Model for Programming and Natural Languages](https://mlsanity.com/paper/2002.08155/) (2020)
- [A Survey on Evaluation of Large Language Models](https://mlsanity.com/paper/2307.03109/) (2023)
- [Gemma: Open Models Based on Gemini Research and Technology](https://mlsanity.com/paper/2403.08295/) (2024)
- [CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation](https://mlsanity.com/paper/2109.00859/) (2021)
- [A Survey on In-context Learning](https://mlsanity.com/paper/2301.00234/) (2022)
- [GraphCodeBERT: Pre-training Code Representations with Data Flow](https://mlsanity.com/paper/2009.08366/) (2020)
- [UniXcoder: Unified Cross-Modal Pre-training for Code Representation](https://mlsanity.com/paper/2203.03850/) (2022)
- [LLM-Adapters: An Adapter Family for Parameter-Efficient Fine-Tuning of Large Language Models](https://mlsanity.com/paper/2304.01933/) (2023)
- [Parameter-Efficient Fine-Tuning Methods for Pretrained Language Models: A Critical Review and Assessment](https://mlsanity.com/paper/2312.12148/) (2023)
- [Fine-tuning Language Models for Factuality](https://mlsanity.com/paper/2311.08401/) (2023)
- [OpenMoE: An Early Effort on Open Mixture-of-Experts Language Models](https://mlsanity.com/paper/2402.01739/) (2024)
- [Efficient Large Language Models: A Survey](https://mlsanity.com/paper/2312.03863/) (2023)
- [Full Parameter Fine-tuning for Large Language Models with Limited Resources](https://mlsanity.com/paper/2306.09782/) (2023)
- [LLMs Cannot Reliably Identify and Reason About Security Vulnerabilities (Yet?): A Comprehensive Evaluation, Framework, and Benchmarks](https://mlsanity.com/paper/2312.12575/) (2023)

## Built on by (most cited first, 30 of 41)

- [From LLMs to LLM-based Agents for Software Engineering: A Survey of Current, Challenges and Future](https://mlsanity.com/paper/2408.02479/) (2024)
- [Taming Various Privilege Escalation in LLM-Based Agent Systems: A Mandatory Access Control Framework](https://mlsanity.com/paper/2601.11893/) (2026)
- [LLMs in Software Security: A Survey of Vulnerability Detection Techniques and Insights](https://mlsanity.com/paper/2502.07049/) (2025)
- [Fraud-R1 : A Multi-Round Benchmark for Assessing the Robustness of LLM Against Augmented Fraud and Phishing Inducements](https://mlsanity.com/paper/2502.12904/) (2025)
- [Testing and Understanding Erroneous Planning in LLM Agents through Synthesized User Inputs](https://mlsanity.com/paper/2404.17833/) (2024)
- [A Survey on Agentic Security: Applications, Threats and Defenses](https://mlsanity.com/paper/2510.06445/) (2025)
- [Measuring and Augmenting Large Language Models for Solving Capture-the-Flag Challenges](https://mlsanity.com/paper/2506.17644/) (2025)
- [Taxonomy, Evaluation and Exploitation of IPI-Centric LLM Agent Defense Frameworks](https://mlsanity.com/paper/2511.15203/) (2025)
- [Aitomia: Your Intelligent Assistant for AI-Driven Atomistic and Quantum Chemical Simulations](https://mlsanity.com/paper/2505.08195/) (2025)
- [VulAgent: Hypothesis-Validation based Multi-Agent Vulnerability Detection](https://mlsanity.com/paper/2509.11523/) (2025)
- [Smart-LLaMA-DPO: Reinforced Large Language Model for Explainable Smart Contract Vulnerability Detection](https://mlsanity.com/paper/2506.18245/) (2025)
- [MAVUL: Multi-Agent Vulnerability Detection via Contextual Reasoning and Interactive Refinement](https://mlsanity.com/paper/2510.00317/) (2025)
- [One Bug, Hundreds Behind: LLMs for Large-Scale Bug Discovery](https://mlsanity.com/paper/2510.14036/) (2025)
- [Logic Meets Magic: LLMs Cracking Smart Contract Vulnerabilities](https://mlsanity.com/paper/2501.07058/) (2025)
- [Efficacy of Various Large Language Models in Generating Smart Contracts](https://mlsanity.com/paper/2407.11019/) (2024)
- [Detecting Various DeFi Price Manipulations with LLM Reasoning](https://mlsanity.com/paper/2502.11521/) (2025)
- [Prompt to Pwn: Automated Exploit Generation for Smart Contracts](https://mlsanity.com/paper/2508.01371/) (2025)
- [SoK: Security and Privacy of AI Agents for Blockchain](https://mlsanity.com/paper/2509.07131/) (2025)
- [Designing LLM-based Multi-Agent Systems for Software Engineering Tasks: Quality Attributes, Design Patterns and Rationale](https://mlsanity.com/paper/2511.08475/) (2025)
- [SymGPT: Auditing Smart Contracts via Combining Symbolic Execution with Large Language Models](https://mlsanity.com/paper/2502.07644/) (2025)
- [Adaptive Plan-Execute Framework for Smart Contract Security Auditing](https://mlsanity.com/paper/2505.15242/) (2025)
- [SAEL: Leveraging Large Language Models with Adaptive Mixture-of-Experts for Smart Contract Vulnerability Detection](https://mlsanity.com/paper/2507.22371/) (2025)
- [LISA Technical Report: An Agentic Framework for Smart Contract Auditing](https://mlsanity.com/paper/2509.24698/) (2025)
- [Connecting Large Language Models with Blockchain: Advancing the Evolution of Smart Contracts from Automation to Intelligence](https://mlsanity.com/paper/2412.02263/) (2024)
- [Experiences with Content Development and Assessment Design in the Era of GenAI](https://mlsanity.com/paper/2503.00081/) (2025)
- [UEChecker: Detecting Unchecked External Call Vulnerabilities in DApps via Graph Analysis](https://mlsanity.com/paper/2508.01343/) (2025)
- [Web3 x AI Agents: Landscape, Integrations, and Foundational Challenges](https://mlsanity.com/paper/2508.02773/) (2025)
- [Towards Secure and Explainable Smart Contract Generation with Security-Aware Group Relative Policy Optimization](https://mlsanity.com/paper/2509.09942/) (2025)
- [Fine-tuning of Large Language Models for Domain-Specific Cybersecurity Knowledge](https://mlsanity.com/paper/2509.25241/) (2025)
- [AutoLogger: A Multi-Agent Framework for the End-to-End Automated Logging](https://mlsanity.com/paper/2511.18528/) (2025)

## Abstract (arXiv)

Smart contracts are decentralized applications built atop blockchains like Ethereum. Recent research has shown that large language models (LLMs) have potential in auditing smart contracts, but the state-of-the-art indicates that even GPT-4 can achieve only 30% precision (when both decision and justification are correct). This is likely because off-the-shelf LLMs were primarily pre-trained on a general text/code corpus and not fine-tuned on the specific domain of Solidity smart contract auditing. In this paper, we propose iAudit, a general framework that combines fine-tuning and LLM-based agents for intuitive smart contract auditing with justifications. Specifically, iAudit is inspired by the observation that expert human auditors first perceive what could be wrong and then perform a detailed analysis of the code to identify the cause. As such, iAudit employs a two-stage fine-tuning approach: it first tunes a Detector model to make decisions and then tunes a Reasoner model to generate causes of vulnerabilities. However, fine-tuning alone faces challenges in accurately identifying the optimal cause of a vulnerability. Therefore, we introduce two LLM-based agents, the Ranker and Critic, to iteratively select and debate the most suitable cause of vulnerability based on the output of the fine-tuned Reasoner model. To evaluate iAudit, we collected a balanced dataset with 1,734 positive and 1,810 negative samples to fine-tune iAudit. We then compared it with traditional fine-tuned models (CodeBERT, GraphCodeBERT, CodeT5, and UnixCoder) as well as prompt learning-based LLMs (GPT4, GPT-3.5, and CodeLlama-13b/34b). On a dataset of 263 real smart contract vulnerabilities, iAudit achieves an F1 score of 91.21% and an accuracy of 91.11%. The causes generated by iAudit achieved a consistency of about 38% compared to the ground truth causes.