SCRUBD: Smart Contracts Reentrancy and Unhandled Exceptions Vulnerability Dataset

Smart Contracts (SCs) handle transactions in the Ethereum blockchain worth millions of United States dollars, making them a lucrative target for attackers seeking to exploit vulnerabilities and steal funds. The Ethereum community has developed a rich set of tools to detect vulnerabilities in SCs, including reentrancy (RE) and unhandled exceptions (UX). A dataset of SCs labelled with vulnerabilities is needed to evaluate the tools' efficacy. Existing SC datasets with labelled vulnerabilities have limitations, such as covering only a limited range of vulnerability scenarios and containing incorrect labels. As a result, there is a lack of a standardized dataset to compare the performances of these tools. SCRUBD aims to fill this gap. We present a dataset of real-world SCs and synthesized SCs labelled with RE and UX. The real-world SC dataset is labelled through crowdsourcing, followed by manual inspection by an expert, and covers both RE and UX vulnerabilities. On the other hand, the synthesized dataset is carefully crafted to cover various RE scenarios only. Using SCRUBD we compared the performance of six popular vulnerability detection tools. Based on our study, we found that Slither outperforms other tools on a crowdsourced dataset in detecting RE vulnerabilities, while Sailfish outperforms other tools on a manually synthesized dataset for detecting RE. For UX vulnerabilities, Slither outperforms all other tools.