# EVMbench: Evaluating AI Agents on Smart Contract Security

arXiv: https://arxiv.org/abs/2603.04915 · HTML: https://mlsanity.com/paper/2603.04915/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 23 references that are in the map.
- First posted on arXiv 5 Mar 2026.

## Builds on (most cited first, 3 of 3)

- [GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks](https://mlsanity.com/paper/2510.04374/) (2025)
- [CVE-Bench: A Benchmark for AI Agents' Ability to Exploit Real-World Web Application Vulnerabilities](https://mlsanity.com/paper/2503.17332/) (2025)
- [BountyBench: Dollar Impact of AI Agent Attackers and Defenders on Real-World Cybersecurity Systems](https://mlsanity.com/paper/2505.15216/) (2025)

## Built on by (most cited first, 7 of 7)

- [Towards Optimal Agentic Architectures for Offensive Security Tasks](https://mlsanity.com/paper/2604.18718/) (2026)
- [Knowdit: Agentic Smart Contract Vulnerability Detection with Auditing Knowledge Summarization](https://mlsanity.com/paper/2603.26270/) (2026)
- [Alignment Contracts for Agentic Security Systems](https://mlsanity.com/paper/2605.00081/) (2026)
- [AISafetyBenchExplorer: A Metric-Aware Catalogue of AI Safety Benchmarks Reveals Fragmented Measurement and Weak Benchmark Governance](https://mlsanity.com/paper/2604.12875/) (2026)
- [CHAINTRIX: A multi-pipeline LLM-augmented framework for automated smart-contract security auditing](https://mlsanity.com/paper/2605.09350/) (2026)
- [CyberChainBench: Can AI Agents Secure Smart Contracts Against Real-World On-Chain Vulnerabilities?](https://mlsanity.com/paper/2606.26216/) (2026)
- [Demystifying Agent Skills for Smart Contract Auditing: Design, Effectiveness, Behavioral Impact](https://mlsanity.com/paper/2609.29454/) (2026)

## Abstract (arXiv)

Smart contracts on public blockchains now manage large amounts of value, and vulnerabilities in these systems can lead to substantial losses. As AI agents become more capable at reading, writing, and running code, it is natural to ask how well they can already navigate this landscape, both in ways that improve security and in ways that might increase risk. We introduce EVMbench, an evaluation that measures the ability of agents to detect, patch, and exploit smart contract vulnerabilities. EVMbench draws on 117 curated vulnerabilities from 40 repositories and, in the most realistic setting, uses programmatic grading based on tests and blockchain state under a local Ethereum execution environment. We evaluate a range of frontier agents and find that they are capable of discovering and exploiting vulnerabilities end-to-end against live blockchain instances. We release code, tasks, and tooling to support continued measurement of these capabilities and future work on security.