# Ethereum Fraud Detection via Joint Transaction Language Model and Graph Representation Learning

arXiv: https://arxiv.org/abs/2409.07494 · HTML: https://mlsanity.com/paper/2409.07494/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 51 references that are in the map.
- First posted on arXiv 9 Sep 2024.

## Builds on (most cited first, 22 of 22)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [A Comprehensive Survey on Graph Neural Networks](https://mlsanity.com/paper/1901.00596/) (2019)
- [node2vec: Scalable Feature Learning for Networks](https://mlsanity.com/paper/1607.00653/) (2016)
- [Graph Neural Networks: A Review of Methods and Applications](https://mlsanity.com/paper/1812.08434/) (2018)
- [DeepWalk: Online Learning of Social Representations](https://mlsanity.com/paper/1403.6652/) (2014)
- [BertGCN: Transductive Text Classification by Combining GCN and BERT](https://mlsanity.com/paper/2105.05727/) (2021)
- [Who Are the Phishers? Phishing Scam Detection on Ethereum via Network Embedding](https://mlsanity.com/paper/1911.09259/) (2019)
- [VGCN-BERT: Augmenting BERT with Graph Embedding for Text Classification](https://mlsanity.com/paper/2004.05707/) (2020)
- [SoK: Decentralized Finance (DeFi)](https://mlsanity.com/paper/2101.08778/) (2021)
- [BERT4ETH: A Pre-trained Transformer for Ethereum Fraud Detection](https://mlsanity.com/paper/2303.18138/) (2023)
- [TTAGN: Temporal Transaction Aggregation Graph Network for Ethereum Phishing Scams Detection](https://mlsanity.com/paper/2204.13442/) (2022)
- [Quantifying Blockchain Extractable Value: How dark is the forest?](https://mlsanity.com/paper/2101.05511/) (2021)
- [Vul-LMGNNs: Fusing language models and online-distilled graph neural networks for code vulnerability detection](https://mlsanity.com/paper/2404.14719/) (2024)
- [High-Frequency Trading on Decentralized On-Chain Exchanges](https://mlsanity.com/paper/2009.14021/) (2020)
- [DAppSCAN: Building Large-Scale Datasets for Smart Contract Weaknesses in DApp Projects](https://mlsanity.com/paper/2305.08456/) (2023)
- [TSGN: Transaction Subgraph Networks Assisting Phishing Detection in Ethereum](https://mlsanity.com/paper/2208.12938/) (2022)
- [On the Just-In-Time Discovery of Profit-Generating Transactions in DeFi Protocols](https://mlsanity.com/paper/2103.02228/) (2021)
- [Blockchain Phishing Scam Detection via Multi-channel Graph Classification](https://mlsanity.com/paper/2108.08456/) (2021)
- [Demystifying DeFi MEV Activities in Flashbots Bundle](https://mlsanity.com/paper/2312.01091/) (2023)

## Built on by (most cited first, 8 of 8)

- [LMAE4Eth: Generalizable and Robust Ethereum Fraud Detection by Exploring Transaction Semantics and Masked Graph Embedding](https://mlsanity.com/paper/2509.03939/) (2025)
- [A Novel Approach to Malicious Code Detection Using CNN-BiLSTM and Feature Fusion](https://mlsanity.com/paper/2410.09401/) (2024)
- [From Past to Present: A Survey of Malicious URL Detection Techniques, Datasets and Code Repositories](https://mlsanity.com/paper/2504.16449/) (2025)
- [Human-Centered LLM-Agent System for Detecting Anomalous Digital Asset Transactions](https://mlsanity.com/paper/2510.20102/) (2025)
- [Transformers in Protein: A Survey](https://mlsanity.com/paper/2505.20098/) (2025)
- [Knowledge-Integrated Representation Learning for Crypto Anomaly Detection under Extreme Label Scarcity; Relational Domain-Logic Integration with Retrieval-Grounded Context and Path-Level Explanations](https://mlsanity.com/paper/2601.12839/) (2026)
- [DLT-Corpus: A Large-Scale Text Collection for the Distributed Ledger Technology Domain](https://mlsanity.com/paper/2602.22045/) (2026)
- [Modeling Decisions in Blockchain Analytics: A Leakage-Aware Evaluation of Tree-Based vs. Sequential Models](https://mlsanity.com/paper/2607.27350/) (2026)

## Abstract (arXiv)

Ethereum faces growing fraud threats. Current fraud detection methods, whether employing graph neural networks or sequence models, fail to consider the semantic information and similarity patterns within transactions. Moreover, these approaches do not leverage the potential synergistic benefits of combining both types of models. To address these challenges, we propose TLMG4Eth that combines a transaction language model with graph-based methods to capture semantic, similarity, and structural features of transaction data in Ethereum. We first propose a transaction language model that converts numerical transaction data into meaningful transaction sentences, enabling the model to learn explicit transaction semantics. Then, we propose a transaction attribute similarity graph to learn transaction similarity information, enabling us to capture intuitive insights into transaction anomalies. Additionally, we construct an account interaction graph to capture the structural information of the account transaction network. We employ a deep multi-head attention network to fuse transaction semantic and similarity embeddings, and ultimately propose a joint training approach for the multi-head attention network and the account interaction graph to obtain the synergistic benefits of both.