# SolBehave: Evaluating the Behavioural Correctness of LLM-Generated Smart Contracts

arXiv: https://arxiv.org/abs/2610.18745 · HTML: https://mlsanity.com/paper/2610.18745/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 36 references that are in the map.
- First posted on arXiv 13 Oct 2026.

## Builds on (most cited first, 12 of 12)

- [LINC: A Neurosymbolic Approach for Logical Reasoning by Combining Language Models with First-Order Logic Provers](https://mlsanity.com/paper/2310.15164/) (2023)
- [Slither: A Static Analysis Framework For Smart Contracts](https://mlsanity.com/paper/1908.09878/) (2019)
- [Manticore: A User-Friendly Symbolic Execution Framework for Binaries and Smart Contracts](https://mlsanity.com/paper/1907.03890/) (2019)
- [Empirical Review of Automated Analysis Tools on 47,587 Ethereum Smart Contracts](https://mlsanity.com/paper/1910.10601/) (2019)
- [Rethinking Smart Contract Fuzzing: Fuzzing With Invocation Ordering and Important Branch Revisiting](https://mlsanity.com/paper/2301.03943/) (2023)
- [LLM4Fuzz: Guided Fuzzing of Smart Contracts with Large Language Models](https://mlsanity.com/paper/2401.11108/) (2024)
- [ItyFuzz: Snapshot-Based Fuzzer for Smart Contract](https://mlsanity.com/paper/2306.17135/) (2023)
- [MuFuzz: Sequence-Aware Mutation and Seed Mask Guidance for Blockchain Smart Contract Fuzzing](https://mlsanity.com/paper/2312.04512/) (2023)
- [Neuro-Symbolic Artificial Intelligence: A Task-Directed Survey in the Black-Box Models Era](https://mlsanity.com/paper/2603.03177/) (2026)
- [Capturing Smart Contract Design with DCR Graphs](https://mlsanity.com/paper/2305.04581/) (2023)
- [HighGuard: Cross-Chain Business Logic Monitoring of Smart Contracts](https://mlsanity.com/paper/2305.08254/) (2023)
- [SolContractEval: A Benchmark for Evaluating Contract-Level Solidity Code Generation](https://mlsanity.com/paper/2509.23824/) (2025)

## Abstract (arXiv)

Large language models increasingly generate Solidity smart contracts that, once deployed, can never be patched, making a single latent fault permanent and exploitable. Yet current evaluations mostly ask only whether generated code compiles. We introduce SolBehave, a benchmark for evaluating the correctness of model-generated contracts at scale, moving beyond ad hoc unit tests. Across 250 real, reproducible Ethereum contracts, we apply three complementary tests of behavioural fidelity: differential fuzzing, historical-transaction replay, and, as our core contribution, synthetic-transaction replay guided by a Dynamic Condition Response (DCR) graph that enumerates the space of legitimate function-call sequences. Differential fuzzing compares each generated function with the original on automatically generated inputs; historical replay compares the whole contract on the transactions it actually received on-chain; DCR-guided replay compares it on sequences of calls that follow its intended usage, including orderings that never occurred on-chain. We evaluate seven state-of-the-art open-weight models after one round of automated repair. Compilation success ranges from 26.4% to 92.7%, but under differential fuzzing only 31.4% of all generated function bodies that compile, pooled over models and samples, are equivalent to the original. Divergence at the contract level is the norm rather than the exception: for six of the seven models, between 13% and 31% of the generated contracts that compile diverge from the original under historical replay, and between 13% and 44% under DCR-guided replay; only Gemma-4-31B stays below 3% on both. Compilation is therefore a necessary condition for correctness and far from a sufficient one.