# Can AI Make Money in Crypto? Measuring the Gap from Backtests to Real Markets

arXiv: https://arxiv.org/abs/2609.34510 · HTML: https://mlsanity.com/paper/2609.34510/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 29 of its 46 references that are in the map.
- First posted on arXiv 28 Sep 2026.

## Builds on (most cited first, 29 of 29)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling](https://mlsanity.com/paper/1803.01271/) (2018)
- [Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecasting](https://mlsanity.com/paper/1912.09363/) (2019)
- [TabNet: Attentive Interpretable Tabular Learning](https://mlsanity.com/paper/1908.07442/) (2019)
- [FinGPT: Open-Source Financial Large Language Models](https://mlsanity.com/paper/2306.06031/) (2023)
- [A Dual-Stage Attention-Based Recurrent Neural Network for Time Series Prediction](https://mlsanity.com/paper/1704.02971/) (2017)
- [TradingAgents: Multi-Agents LLM Financial Trading Framework](https://mlsanity.com/paper/2412.20138/) (2024)
- [FinMem: A Performance-Enhanced LLM Trading Agent with Layered Memory and Character Design](https://mlsanity.com/paper/2311.13743/) (2023)
- [FinBen: A Holistic Financial Benchmark for Large Language Models](https://mlsanity.com/paper/2402.12659/) (2024)
- [A Multimodal Foundation Agent for Financial Trading: Tool-Augmented, Diversified, and Generalist](https://mlsanity.com/paper/2402.18485/) (2024)
- [AdaRNN: Adaptive Learning and Forecasting of Time Series](https://mlsanity.com/paper/2108.04443/) (2021)
- [Qlib: An AI-oriented Quantitative Investment Platform](https://mlsanity.com/paper/2009.11189/) (2020)
- [INVESTORBENCH: A Benchmark for Financial Decision-Making Tasks with LLM-based Agent](https://mlsanity.com/paper/2412.18174/) (2024)
- [HIST: A Graph-based Framework for Stock Trend Forecasting via Mining Concept-Oriented Shared Information](https://mlsanity.com/paper/2110.13716/) (2021)
- [StockBench: Can LLM Agents Trade Stocks Profitably In Real-world Markets?](https://mlsanity.com/paper/2510.02209/) (2025)
- [Learning Multiple Stock Trading Patterns with Temporal Routing Adaptor and Optimal Transport](https://mlsanity.com/paper/2106.12950/) (2021)
- [AI-Trader: Benchmarking Autonomous Agents in Real-Time Financial Markets](https://mlsanity.com/paper/2512.10971/) (2025)
- [Time Travel is Cheating: Going Live with DeepFund for Real-Time Fund Investment Benchmarking](https://mlsanity.com/paper/2505.11065/) (2025)
- [LiveTradeBench: Seeking Real-World Alpha with Large Language Models](https://mlsanity.com/paper/2511.03628/) (2025)
- [DoubleEnsemble: A New Ensemble Method Based on Sample Reweighting and Feature Selection for Financial Data Analysis](https://mlsanity.com/paper/2010.01265/) (2020)
- [MacroHFT: Memory Augmented Context-aware Reinforcement Learning On High Frequency Trading](https://mlsanity.com/paper/2406.14537/) (2024)
- [Temporally Correlated Task Scheduling for Sequence Learning](https://mlsanity.com/paper/2007.05290/) (2020)
- [ADD: Augmented Disentanglement Distillation Framework for Improving Stock Trend Forecasting](https://mlsanity.com/paper/2012.06289/) (2020)
- [Instance-wise Graph-based Framework for Multivariate Time Series Forecasting](https://mlsanity.com/paper/2109.06489/) (2021)

## Abstract (arXiv)

AI-based trading methods have rapidly evolved from machine learning and reinforcement learning to large language models (LLMs) and trading agents, yet their performance is still predominantly assessed through historical backtesting. Such evaluations provide limited evidence of whether a method can generalize to unseen future markets or whether its backtested performance can be sustained in realistic trading frictions (e.g., latency, slippage, liquidity constraints, and market impact). We present a unified benchmark that evaluates representative machine learning, reinforcement learning, LLM-based, and agent-based trading methods in cryptocurrency markets through three progressively more realistic stages: historical backtesting, prospective exchange-based paper trading, and real-money live trading. These stages jointly increase temporal realism by moving from historical to unseen future markets, and execution realism by moving from offline simulation toward live trading. This protocol enables us to quantify the backtest-to-realization gap, identify when performance begins to deteriorate, and compare how this gap differs across major classes of AI trading methods. We further provide a unified open-source system supporting all three evaluation stages, together with a public platform that continuously updates benchmark results. Code is available at https://github.com/Starlien95/Awesome-TradingAI.