mlsanityData to 7 Oct 2026

BERT4ETH: A Pre-trained Transformer for Ethereum Fraud Detection

arXiv 2303.18138 ↗First posted 29 Mar 2023cs.CR cs.LGcited by 19

Not on the map. Why?

Abstract (arXiv)

As various forms of fraud proliferate on Ethereum, it is imperative to safeguard against these malicious activities to protect susceptible users from being victimized. While current studies solely rely on graph-based fraud detection approaches, it is argued that they may not be well-suited for dealing with highly repetitive, skew-distributed and heterogeneous Ethereum transactions. To address these challenges, we propose BERT4ETH, a universal pre-trained Transformer encoder that serves as an account representation extractor for detecting various fraud behaviors on Ethereum. BERT4ETH features the superior modeling capability of Transformer to capture the dynamic sequential patterns inherent in Ethereum transactions, and addresses the challenges of pre-training a BERT model for Ethereum with three practical and effective strategies, namely repetitiveness reduction, skew alleviation and heterogeneity modeling. Our empirical evaluation demonstrates that BERT4ETH outperforms state-of-the-art methods with significant enhancements in terms of the phishing account detection and de-anonymization tasks. The code for BERT4ETH is available at: https://github.com/git-disl/BERT4ETH.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Builds on 25 of its 38 references that are in the map. First posted on arXiv 29 Mar 2023.

Builds on

25 of 38 references.

References · 25

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  5. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  6. 2017Graph Attention Networkscited by 8,854
  7. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  8. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  9. 2017Inductive Representation Learning on Large Graphscited by 6,284
  10. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,801
  11. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  12. 2014DeepWalk: Online Learning of Social Representationscited by 1,543
13 more
  1. 2019Who Are the Phishers? Phishing Scam Detection on Ethereum via Network Embeddingcited by 38
  2. 2019T-EDGE: Temporal WEighted MultiDiGraph Embedding for Ethereum Transaction Network Analysiscited by 19
  3. 2022TTAGN: Temporal Transaction Aggregation Graph Network for Ethereum Phishing Scams Detectioncited by 18
  4. 2022Behavior-aware Account De-anonymization on Ethereum Interaction Graphcited by 17
  5. 2018Learning Role-based Graph Embeddingscited by 14
  6. 2020Fast Sequence-Based Embedding with Diffusion Graphscited by 14
  7. 2019Flash Boys 2.0: Frontrunning, Transaction Reordering, and Consensus Instability in Decentralized Exchangescited by 11
  8. 2020Blockchain is Watching You: Profiling and Deanonymizing Ethereum Userscited by 11
  9. 2020Modeling and Understanding Ethereum Transaction Records via a Complex Network Approachcited by 11
  10. 2021Identity Inference on Blockchain using Graph Neural Networkcited by 9
  11. 2021Detecting and Quantifying Wash Trading on Decentralized Cryptocurrency Exchangescited by 4
  12. 2022Sequence-Based Target Coin Prediction for Cryptocurrency Pump-and-Dumpcited by 4
  13. 2018IcoRating: A Deep-Learning System for Scam ICO Identificationcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 8; 2025: 5; 2026: 3.

19 papers.

Sort:

By area:

  1. 2024Vaccine: Perturbation-aware Alignment for Large Language Models against Harmful Fine-tuning Attackcited by 113
  2. 2024Lisa: Lazy Safety Alignment for Large Language Models against Harmful Fine-tuning Attackcited by 89
  3. 2024Harmful Fine-tuning Attacks and Defenses for Large Language Models: A Surveycited by 86
  4. 2023Large Language Model-Powered Smart Contract Vulnerability Detection: New Perspectivescited by 48
  5. 2024MegaAgent: A Large-Scale Autonomous LLM-based Multi-Agent System Without Predefined SOPscited by 45
  6. 2024Ethereum Fraud Detection via Joint Transaction Language Model and Graph Representation Learningcited by 8
  7. 2025LMAE4Eth: Generalizable and Robust Ethereum Fraud Detection by Exploring Transaction Semantics and Masked Graph Embeddingcited by 3
  8. 2023Effective Illicit Account Detection on Large Cryptocurrency MultiGraphscited by 2
  9. 2024BlockScan: Detecting Anomalies in Blockchain Transactionscited by 2
  10. 2023Leveraging Deep Learning and Online Source Sentiment for Financial Portfolio Managementcited by 1
  11. 2024Enhancing Ethereum Fraud Detection via Generative and Contrastive Self-supervisioncited by 1
  12. 2024MEGA: Message Passing Neural Networks for Multigraphs with EdGe Attributescited by 1
7 more
  1. 2025FraudTransformer: Time-Aware GPT for Transaction Fraud Detectioncited by 1
  2. 2025Large Language Model based Smart Contract Auditing with LLMBugScannercited by 1
  3. 2025Towards Unified Approaches in Self-Supervised Event Stream Modeling: Progress and Prospectscited by 0
  4. 2025Linking Cryptoasset Attribution Tags to Knowledge Graph Entities: An LLM-based Approachcited by 0
  5. 2026NEST: Nested Event Stream Transformer for Sequences of Multisetscited by 0
  6. 2026MELT: A Behavioral Trace Dataset for High-Risk Memecoin Launch Detectioncited by 0
  7. 2026ETH-TraceBench: A Large-Scale Event-Stream Benchmark for Ethereum DeFi under Temporal, Protocol, and Contract Shiftcited by 0