# Smart-LLaMA: Two-Stage Post-Training of Large Language Models for Smart Contract Vulnerability Detection and Explanation

arXiv: https://arxiv.org/abs/2411.06221 · HTML: https://mlsanity.com/paper/2411.06221/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 50 references that are in the map.
- First posted on arXiv 9 Nov 2024.

## Builds on (most cited first, 26 of 26)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Qwen2 Technical Report](https://mlsanity.com/paper/2407.10671/) (2024)
- [LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models](https://mlsanity.com/paper/2403.13372/) (2024)
- [CodeBERT: A Pre-Trained Model for Programming and Natural Languages](https://mlsanity.com/paper/2002.08155/) (2020)
- [CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation](https://mlsanity.com/paper/2109.00859/) (2021)
- [A Comprehensive Overview of Large Language Models](https://mlsanity.com/paper/2307.06435/) (2023)
- [GraphCodeBERT: Pre-training Code Representations with Data Flow](https://mlsanity.com/paper/2009.08366/) (2020)
- [The Adverse Effects of Code Duplication in Machine Learning Models of Code](https://mlsanity.com/paper/1812.06469/) (2018)
- [Slither: A Static Analysis Framework For Smart Contracts](https://mlsanity.com/paper/1908.09878/) (2019)
- [GPTScan: Detecting Logic Vulnerabilities in Smart Contracts by Combining GPT with Program Analysis](https://mlsanity.com/paper/2308.03314/) (2023)
- [Securify: Practical Security Analysis of Smart Contracts](https://mlsanity.com/paper/1806.01143/) (2018)
- [Large Language Model-Powered Smart Contract Vulnerability Detection: New Perspectives](https://mlsanity.com/paper/2310.01152/) (2023)
- [When ChatGPT Meets Smart Contract Vulnerability Detection: How Far Are We?](https://mlsanity.com/paper/2309.05520/) (2023)
- [Manticore: A User-Friendly Symbolic Execution Framework for Binaries and Smart Contracts](https://mlsanity.com/paper/1907.03890/) (2019)
- [Do you still need a manual smart contract audit?](https://mlsanity.com/paper/2306.12338/) (2023)
- [A Survey on Ethereum Systems Security: Vulnerabilities, Attacks and Defenses](https://mlsanity.com/paper/1908.04507/) (2019)
- [Smart Contract Vulnerability Detection: From Pure Neural Network to Interpretable Graph Feature and Expert Pattern Fusion](https://mlsanity.com/paper/2106.09282/) (2021)
- [SmartBugs: A Framework to Analyze Solidity Smart Contracts](https://mlsanity.com/paper/2007.04771/) (2020)
- [Efficient Avoidance of Vulnerabilities in Auto-completed Smart Contract Code Using Vulnerability-constrained Decoding](https://mlsanity.com/paper/2309.09826/) (2023)
- [Rethinking Smart Contract Fuzzing: Fuzzing With Invocation Ordering and Important Branch Revisiting](https://mlsanity.com/paper/2301.03943/) (2023)
- [Security Analysis Methods on Ethereum Smart Contract Vulnerabilities: A Survey](https://mlsanity.com/paper/1908.08605/) (2019)
- [Improving Smart Contract Security with Contrastive Learning-based Vulnerability Detection](https://mlsanity.com/paper/2404.17839/) (2024)
- [Blockchain-based Smart Contracts: A Systematic Mapping Study](https://mlsanity.com/paper/1710.06372/) (2017)
- [SmartEmbed: A Tool for Clone and Bug Detection in Smart Contracts through Structural Code Embedding](https://mlsanity.com/paper/1908.08615/) (2019)
- [EASYFLOW: Keep Ethereum Away From Overflow](https://mlsanity.com/paper/1811.03814/) (2018)

## Built on by (most cited first, 8 of 8)

- [Smart-LLaMA-DPO: Reinforced Large Language Model for Explainable Smart Contract Vulnerability Detection](https://mlsanity.com/paper/2506.18245/) (2025)
- [Towards Practical Defect-Focused Automated Code Review](https://mlsanity.com/paper/2505.17928/) (2025)
- [SAEL: Leveraging Large Language Models with Adaptive Mixture-of-Experts for Smart Contract Vulnerability Detection](https://mlsanity.com/paper/2507.22371/) (2025)
- [Towards Secure and Explainable Smart Contract Generation with Security-Aware Group Relative Policy Optimization](https://mlsanity.com/paper/2509.09942/) (2025)
- [Towards Automated Smart Contract Generation: Evaluation, Benchmarking, and Retrieval-Augmented Repair](https://mlsanity.com/paper/2503.01098/) (2025)
- [Benchmarking Zero-Shot Reasoning Approaches for Error Detection in Solidity Smart Contracts](https://mlsanity.com/paper/2603.13239/) (2026)
- [Decoupled Smart Contract Audits: Lightweight LLM Framework via Distillation and Aggregation](https://mlsanity.com/paper/2606.03128/) (2026)
- [Towards Secure Cloud-Native Computing: Unveiling Kubernetes Misconfigurations with Large Language Models](https://mlsanity.com/paper/2609.20834/) (2026)

## Abstract (arXiv)

With the rapid development of blockchain technology, smart contract security has become a critical challenge. Existing smart contract vulnerability detection methods face three main issues: (1) Insufficient quality of datasets, lacking detailed explanations and precise vulnerability locations. (2) Limited adaptability of large language models (LLMs) to the smart contract domain, as most LLMs are pre-trained on general text data but minimal smart contract-specific data. (3) Lack of high-quality explanations for detected vulnerabilities, as existing methods focus solely on detection without clear explanations. These limitations hinder detection performance and make it harder for developers to understand and fix vulnerabilities quickly, potentially leading to severe financial losses. To address these problems, we propose Smart-LLaMA, an advanced detection method based on the LLaMA language model. First, we construct a comprehensive dataset covering four vulnerability types with labels, detailed explanations, and precise vulnerability locations. Second, we introduce Smart Contract-Specific Continual Pre-Training, using raw smart contract data to enable the LLM to learn smart contract syntax and semantics, enhancing their domain adaptability. Furthermore, we propose Explanation-Guided Fine-Tuning, which fine-tunes the LLM using paired vulnerable code and explanations, enabling both vulnerability detection and reasoned explanations. We evaluate explanation quality through LLM and human evaluation, focusing on Correctness, Completeness, and Conciseness. Experimental results show that Smart-LLaMA outperforms state-of-the-art baselines, with average improvements of 6.49% in F1 score and 3.78% in accuracy, while providing reliable explanations.