# Blockchain Large Language Models

arXiv: https://arxiv.org/abs/2304.12749 · HTML: https://mlsanity.com/paper/2304.12749/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 72 references that are in the map.
- First posted on arXiv 25 Apr 2023.

## Builds on (most cited first, 29 of 29)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [Distributed Representations of Sentences and Documents](https://mlsanity.com/paper/1405.4053/) (2014)
- [Slither: A Static Analysis Framework For Smart Contracts](https://mlsanity.com/paper/1908.09878/) (2019)
- [Securify: Practical Security Analysis of Smart Contracts](https://mlsanity.com/paper/1806.01143/) (2018)
- [Manticore: A User-Friendly Symbolic Execution Framework for Binaries and Smart Contracts](https://mlsanity.com/paper/1907.03890/) (2019)
- [SoK: Decentralized Finance (DeFi) Attacks](https://mlsanity.com/paper/2208.13035/) (2022)
- [Finding The Greedy, Prodigal, and Suicidal Contracts at Scale](https://mlsanity.com/paper/1802.06038/) (2018)
- [sFuzz: An Efficient Adaptive Fuzzer for Solidity Smart Contracts](https://mlsanity.com/paper/2004.08563/) (2020)
- [Sereum: Protecting Existing Smart Contracts Against Re-Entrancy Attacks](https://mlsanity.com/paper/1812.05934/) (2018)
- [Quantifying Blockchain Extractable Value: How dark is the forest?](https://mlsanity.com/paper/2101.05511/) (2021)
- [The Art of The Scam: Demystifying Honeypots in Ethereum Smart Contracts](https://mlsanity.com/paper/1902.06976/) (2019)
- [eThor: Practical and Provably Sound Static Analysis of Ethereum Smart Contracts](https://mlsanity.com/paper/2005.06227/) (2020)
- [Consensus in the Age of Blockchains](https://mlsanity.com/paper/1711.03936/) (2017)
- [Harvey: A Greybox Fuzzer for Smart Contracts](https://mlsanity.com/paper/1905.06944/) (2019)
- [High-Frequency Trading on Decentralized On-Chain Exchanges](https://mlsanity.com/paper/2009.14021/) (2020)
- [SAILFISH: Vetting Smart Contract State-Inconsistency Bugs in Seconds](https://mlsanity.com/paper/2104.08638/) (2021)
- [Attacking the DeFi Ecosystem with Flash Loans for Fun and Profit](https://mlsanity.com/paper/2003.03810/) (2020)
- [Learning Scalable and Precise Representation of Program Semantics](https://mlsanity.com/paper/1905.05251/) (2019)
- [On the Just-In-Time Discovery of Profit-Generating Transactions in DeFi Protocols](https://mlsanity.com/paper/2103.02228/) (2021)
- [DeFiRanger: Detecting Price Manipulation Attacks on DeFi Applications](https://mlsanity.com/paper/2104.15068/) (2021)
- [Oracle-Supported Dynamic Exploit Generation for Smart Contracts](https://mlsanity.com/paper/1909.06605/) (2019)
- [Dynamic Vulnerability Detection on Smart Contracts Using Machine Learning](https://mlsanity.com/paper/2102.07420/) (2021)
- [EtherSolve: Computing an Accurate Control-Flow Graph from Ethereum Bytecode](https://mlsanity.com/paper/2103.09113/) (2021)
- [EthIR: A Framework for High-Level Analysis of Ethereum Bytecode](https://mlsanity.com/paper/1805.07208/) (2018)

## Built on by (most cited first, 17 of 17)

- [LLM4Vuln: A Unified Evaluation Framework for Decoupling and Enhancing LLMs' Vulnerability Reasoning](https://mlsanity.com/paper/2401.16185/) (2024)
- [Frontier AI's Impact on the Cybersecurity Landscape](https://mlsanity.com/paper/2504.05408/) (2025)
- [Byzantine-Robust Decentralized Coordination of LLM Agents](https://mlsanity.com/paper/2507.14928/) (2025)
- [AI Agent Smart Contract Exploit Generation](https://mlsanity.com/paper/2507.05558/) (2025)
- [BC4LLM: Trusted Artificial Intelligence When Blockchain Meets Large Language Models](https://mlsanity.com/paper/2310.06278/) (2023)
- [Detecting Various DeFi Price Manipulations with LLM Reasoning](https://mlsanity.com/paper/2502.11521/) (2025)
- [Comparing Uncertainty Measurement and Mitigation Methods for Large Language Models: A Systematic Review](https://mlsanity.com/paper/2504.18346/) (2025)
- [Large Language Models for Cryptocurrency Transaction Analysis: A Bitcoin Case Study](https://mlsanity.com/paper/2501.18158/) (2025)
- [Machine Learning for Blockchain Data Analysis: Progress and Opportunities](https://mlsanity.com/paper/2404.18251/) (2024)
- [BlockScan: Detecting Anomalies in Blockchain Transactions](https://mlsanity.com/paper/2410.04039/) (2024)
- [Blockchain Meets LLMs: A Living Survey on Bidirectional Integration](https://mlsanity.com/paper/2411.16809/) (2024)
- [TxRay: Agentic Postmortem of Live Blockchain Attacks](https://mlsanity.com/paper/2602.01317/) (2026)
- [Generative AI-enabled Blockchain Networks: Fundamentals, Applications, and Case Study](https://mlsanity.com/paper/2401.15625/) (2024)
- [Crypto x AI, AI x Crypto: A Survey](https://mlsanity.com/paper/2606.13892/) (2026)
- [Linking Cryptoasset Attribution Tags to Knowledge Graph Entities: An LLM-based Approach](https://mlsanity.com/paper/2502.10453/) (2025)
- [LOCARD: An Agentic Framework for Blockchain Forensics](https://mlsanity.com/paper/2604.04211/) (2026)
- [ETH-TraceBench: A Large-Scale Event-Stream Benchmark for Ethereum DeFi under Temporal, Protocol, and Contract Shift](https://mlsanity.com/paper/2609.23659/) (2026)

## Abstract (arXiv)

This paper presents a dynamic, real-time approach to detecting anomalous blockchain transactions. The proposed tool, BlockGPT, generates tracing representations of blockchain activity and trains from scratch a large language model to act as a real-time Intrusion Detection System. Unlike traditional methods, BlockGPT is designed to offer an unrestricted search space and does not rely on predefined rules or patterns, enabling it to detect a broader range of anomalies. We demonstrate the effectiveness of BlockGPT through its use as an anomaly detection tool for Ethereum transactions. In our experiments, it effectively identifies abnormal transactions among a dataset of 68M transactions and has a batched throughput of 2284 transactions per second on average. Our results show that, BlockGPT identifies abnormal transactions by ranking 49 out of 124 attacks among the top-3 most abnormal transactions interacting with their victim contracts. This work makes contributions to the field of blockchain transaction analysis by introducing a custom data encoding compatible with the transformer architecture, a domain-specific tokenization technique, and a tree encoding method specifically crafted for the Ethereum Virtual Machine (EVM) trace representation.