mlsanityData to 14 Oct 2026

An Empirical Study of AI-based Smart Contract Creation

arXiv 2308.02955 ↗First posted 5 Aug 2023cs.SE cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

The introduction of large language models (LLMs) like ChatGPT and Google Palm2 for smart contract generation seems to be the first well-established instance of an AI pair programmer. LLMs have access to a large number of open-source smart contracts, enabling them to utilize more extensive code in Solidity than other code generation tools. Although the initial and informal assessments of LLMs for smart contract generation are promising, a systematic evaluation is needed to explore the limits and benefits of these models. The main objective of this study is to assess the quality of generated code provided by LLMs for smart contracts. We also aim to evaluate the impact of the quality and variety of input parameters fed to LLMs. To achieve this aim, we created an experimental setup for evaluating the generated code in terms of validity, correctness, and efficiency. Our study finds crucial evidence of security bugs getting introduced in the generated smart contracts as well as the overall quality and correctness of the code getting impacted. However, we also identified the areas where it can be improved. The paper also proposes several potential research directions to improve the process, quality and safety of generated smart contract codes.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 20 of its 44 references that are in the map. First posted on arXiv 5 Aug 2023.

Builds on

20 of 44 references.

References · 20

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  5. 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,310
  6. 2023PaLM 2 Technical Reportcited by 1,162
  7. 2021CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generationcited by 906
  8. 2022CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesiscited by 885
  9. 2020GraphCodeBERT: Pre-training Code Representations with Data Flowcited by 609
  10. 2022InCoder: A Generative Model for Code Infilling and Synthesiscited by 498
  11. 2021Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributionscited by 431
  12. 2021Unified Pre-training for Program Understanding and Generationcited by 366
8 more
  1. 2023Exploring the Benefits of Training Expert Language Models over Instruction Tuningcited by 80
  2. 2023Evaluating the Code Quality of AI-Assisted Code Generation Tools: An Empirical Study on GitHub Copilot, Amazon CodeWhisperer, and ChatGPTcited by 71
  3. 2023On the Robustness of Code Generation Techniques: An Empirical Study on GitHub Copilotcited by 64
  4. 2023CodeLMSec Benchmark: Systematically Evaluating and Finding Security Vulnerabilities in Black-Box Code Language Modelscited by 55
  5. 2023Learning Deep Semantics for Test Completioncited by 31
  6. 2022Targeted Phishing Campaigns using Large Scale Language Modelscited by 23
  7. 2019Harvey: A Greybox Fuzzer for Smart Contractscited by 10
  8. 2022Lessons Learned from Blockchain Applications of Trusted Execution Environments and Implications for Future Researchcited by 3

Built on by

Cited by year

Citing papers per year: 2024: 3; 2025: 1.

4 papers.

  1. 2024Efficacy of Various Large Language Models in Generating Smart Contractscited by 4
  2. 2024Teaching Machines to Code: Smart Contract Translation with LLMscited by 3
  3. 2024LogBabylon: A Unified Framework for Cross-Log File Integration and Analysiscited by 1
  4. 2025On LLM-Assisted Generation of Smart Contracts from Business Processescited by 1