# A Multi-Modal Transformer-based Code Summarization Approach for Smart Contracts

arXiv: https://arxiv.org/abs/2103.07164 · HTML: https://mlsanity.com/paper/2103.07164/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Mar 2021.

## Built on by (most cited first, 16 of 16)

- [Exploring and Unleashing the Power of Large Language Models in Automated Code Translation](https://mlsanity.com/paper/2404.14646/) (2024)
- [Automatic Code Summarization via ChatGPT: How Far Are We?](https://mlsanity.com/paper/2305.12865/) (2023)
- [A Survey on Machine Learning Techniques for Source Code Analysis](https://mlsanity.com/paper/2110.09610/) (2021)
- [Deep Learning for Code Intelligence: Survey, Benchmark and Toolkit](https://mlsanity.com/paper/2401.00288/) (2023)
- [An Extractive-and-Abstractive Framework for Source Code Summarization](https://mlsanity.com/paper/2206.07245/) (2022)
- [M2TS: Multi-Scale Multi-Modal Approach Based on Transformer for Source Code Summarization](https://mlsanity.com/paper/2203.09707/) (2022)
- [ProCQA: A Large-scale Community-based Programming Question Answering Dataset for Code Search](https://mlsanity.com/paper/2403.16702/) (2024)
- [Automating Code-Related Tasks Through Transformers: The Impact of Pre-training](https://mlsanity.com/paper/2302.04048/) (2023)
- [ESALE: Enhancing Code-Summary Alignment Learning for Source Code Summarization](https://mlsanity.com/paper/2407.01646/) (2024)
- [Improving Stack Overflow question title generation with copying enhanced CodeBERT model and bi-modal information](https://mlsanity.com/paper/2109.13073/) (2021)
- [Abstract Syntax Tree for Programming Language Understanding and Representation: How Far Are We?](https://mlsanity.com/paper/2312.00413/) (2023)
- [The Role of Transformer Models in Advancing Blockchain Technology: A Systematic Survey](https://mlsanity.com/paper/2409.02139/) (2024)
- [Diverse Title Generation for Stack Overflow Posts with Multiple Sampling Enhanced Transformer](https://mlsanity.com/paper/2208.11523/) (2022)
- [An Exploratory Literature Study on Sharing and Energy Use of Language Models for Source Code](https://mlsanity.com/paper/2307.02443/) (2023)
- [Analysing the Behaviour of Tree-Based Neural Networks in Regression Tasks](https://mlsanity.com/paper/2406.11437/) (2024)
- [evalSmarT: An LLM-Based Framework for Evaluating Smart Contract Generated Comments](https://mlsanity.com/paper/2507.20774/) (2025)

## Abstract (arXiv)

Code comment has been an important part of computer programs, greatly facilitating the understanding and maintenance of source code. However, high-quality code comments are often unavailable in smart contracts, the increasingly popular programs that run on the blockchain. In this paper, we propose a Multi-Modal Transformer-based (MMTrans) code summarization approach for smart contracts. Specifically, the MMTrans learns the representation of source code from the two heterogeneous modalities of the Abstract Syntax Tree (AST), i.e., Structure-based Traversal (SBT) sequences and graphs. The SBT sequence provides the global semantic information of AST, while the graph convolution focuses on the local details. The MMTrans uses two encoders to extract both global and local semantic information from the two modalities respectively, and then uses a joint decoder to generate code comments. Both the encoders and the decoder employ the multi-head attention structure of the Transformer to enhance the ability to capture the long-range dependencies between code tokens. We build a dataset with over 300K <method, comment> pairs of smart contracts, and evaluate the MMTrans on it. The experimental results demonstrate that the MMTrans outperforms the state-of-the-art baselines in terms of four evaluation metrics by a substantial margin, and can generate higher quality comments.