# An end-to-end agentic pipeline for smart contract translation and quality evaluation

arXiv: https://arxiv.org/abs/2602.13808 · HTML: https://mlsanity.com/paper/2602.13808/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use.
- Builds on 15 of its 27 references that are in the map.
- First posted on arXiv 14 Feb 2026.

## Placed in

- LLM agents and tool use / [Agent threats, guardrails and governance](https://mlsanity.com/agents/agent-threats-guardrails-and-governance/)

## Builds on (most cited first, 15 of 15)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Program Synthesis with Large Language Models](https://mlsanity.com/paper/2108.07732/) (2021)
- [Code Llama: Open Foundation Models for Code](https://mlsanity.com/paper/2308.12950/) (2023)
- [Slither: A Static Analysis Framework For Smart Contracts](https://mlsanity.com/paper/1908.09878/) (2019)
- [A Survey on Ethereum Systems Security: Vulnerabilities, Attacks and Defenses](https://mlsanity.com/paper/1908.04507/) (2019)
- [An Overview on Smart Contracts: Challenges, Advances and Platforms](https://mlsanity.com/paper/1912.10370/) (2019)
- [SmartInv: Multimodal Learning for Smart Contract Invariant Inference](https://mlsanity.com/paper/2411.09217/) (2024)
- [Guiding LLM-based Smart Contract Generation with Finite State Machine](https://mlsanity.com/paper/2505.08542/) (2025)
- [Designing Secure Ethereum Smart Contracts: A Finite State Machine Based Approach](https://mlsanity.com/paper/1711.09327/) (2017)
- [Detecting Buggy Contracts via Smart Testing](https://mlsanity.com/paper/2409.04597/) (2024)
- [Decentralized Finance: Protocols, Risks, and Governance](https://mlsanity.com/paper/2312.01018/) (2023)

## Abstract (arXiv)

We present an end-to-end framework for systematic evaluation of LLM-generated smart contracts from natural-language specifications. The system parses contractual text into structured schemas, generates Solidity code, and performs automated quality assessment through compilation and security checks. Using CrewAI-style agent teams with iterative refinement, the pipeline produces structured artifacts with full provenance metadata. Quality is measured across five dimensions, including functional completeness, variable fidelity, state-machine correctness, business-logic fidelity, and code quality aggregated into composite scores. The framework supports paired evaluation against ground-truth implementations, quantifying alignment and identifying systematic error modes such as logic omissions and state transition inconsistencies. This provides a reproducible benchmark for empirical research on smart contract synthesis quality and supports extensions to formal verification and compliance checking.