# Time Travel is Cheating: Going Live with DeepFund for Real-Time Fund Investment Benchmarking

arXiv: https://arxiv.org/abs/2505.11065 · HTML: https://mlsanity.com/paper/2505.11065/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 45 of its 79 references that are in the map.
- First posted on arXiv 16 May 2025.

## Builds on (most cited first, 30 of 45)

- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023)
- [LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code](https://mlsanity.com/paper/2403.07974/) (2024)
- [Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference](https://mlsanity.com/paper/2403.04132/) (2024)
- [Large Language Models](https://mlsanity.com/paper/2307.05782/) (2023)
- [AFlow: Automating Agentic Workflow Generation](https://mlsanity.com/paper/2410.10762/) (2024)
- [TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Finance](https://mlsanity.com/paper/2105.07624/) (2021)
- [Agent Workflow Memory](https://mlsanity.com/paper/2409.07429/) (2024)
- [Check Your Facts and Try Again: Improving Large Language Models with External Knowledge and Automated Feedback](https://mlsanity.com/paper/2302.12813/) (2023)
- [FinanceBench: A New Benchmark for Financial Question Answering](https://mlsanity.com/paper/2311.11944/) (2023)
- [Large Language Models in Finance: A Survey](https://mlsanity.com/paper/2311.10723/) (2023)
- [TradingAgents: Multi-Agents LLM Financial Trading Framework](https://mlsanity.com/paper/2412.20138/) (2024)
- [FinMem: A Performance-Enhanced LLM Trading Agent with Layered Memory and Character Design](https://mlsanity.com/paper/2311.13743/) (2023)
- [FinBen: A Holistic Financial Benchmark for Large Language Models](https://mlsanity.com/paper/2402.12659/) (2024)
- [A Multimodal Foundation Agent for Financial Trading: Tool-Augmented, Diversified, and Generalist](https://mlsanity.com/paper/2402.18485/) (2024)
- [Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systems](https://mlsanity.com/paper/2504.01990/) (2025)
- [ForecastBench: A Dynamic Benchmark of AI Forecasting Capabilities](https://mlsanity.com/paper/2409.19839/) (2024)
- [Benchmarking Benchmark Leakage in Large Language Models](https://mlsanity.com/paper/2404.18824/) (2024)
- [Revolutionizing Finance with LLMs: An Overview of Applications and Insights](https://mlsanity.com/paper/2401.11641/) (2024)
- [FinRL: A Deep Reinforcement Learning Library for Automated Stock Trading in Quantitative Finance](https://mlsanity.com/paper/2011.09607/) (2020)
- [The Dawn of Natural Language to SQL: Are We Fully Ready?](https://mlsanity.com/paper/2406.01265/) (2024)
- [Benchmark Self-Evolving: A Multi-Agent Framework for Dynamic LLM Evaluation](https://mlsanity.com/paper/2402.11443/) (2024)
- [Large language models for automated scholarly paper review: A survey](https://mlsanity.com/paper/2501.10326/) (2025)
- [A Survey of Text-to-SQL in the Era of LLMs: Where are we, and where are we going?](https://mlsanity.com/paper/2408.05109/) (2024)
- [FinTextQA: A Dataset for Long-form Financial Question Answering](https://mlsanity.com/paper/2405.09980/) (2024)
- [Alpha-SQL: Zero-Shot Text-to-SQL using Monte Carlo Tree Search](https://mlsanity.com/paper/2502.17248/) (2025)
- [ConStat: Performance-Based Contamination Detection in Large Language Models](https://mlsanity.com/paper/2405.16281/) (2024)
- [NL2SQL-BUGs: A Benchmark for Detecting Semantic Errors in NL2SQL Translation](https://mlsanity.com/paper/2503.11984/) (2025)
- [Assessing Look-Ahead Bias in Stock Return Predictions Generated By GPT Sentiment Analysis](https://mlsanity.com/paper/2309.17322/) (2023)
- [QuantAgent: Seeking Holy Grail in Trading by Self-Improving Large Language Model](https://mlsanity.com/paper/2402.03755/) (2024)

## Built on by (most cited first, 21 of 21)

- [A Survey of Reasoning and Agentic Systems in Time Series with Large Language Models](https://mlsanity.com/paper/2509.11575/) (2025)
- [LiveTradeBench: Seeking Real-World Alpha with Large Language Models](https://mlsanity.com/paper/2511.03628/) (2025)
- [InfoPO: Information-Driven Policy Optimization for User-Centric Agents](https://mlsanity.com/paper/2603.00656/) (2026)
- [From Knowing to Doing: A Memory-Controlled Benchmark for LLM Trading Agents on Stock Markets](https://mlsanity.com/paper/2605.28359/) (2026)
- [Alpha-R1: Alpha Screening with LLM Reasoning via Reinforcement Learning](https://mlsanity.com/paper/2512.23515/) (2025)
- [Advancing Event Forecasting through Massive Training of Large Language Models: Challenges, Solutions, and Broader Impacts](https://mlsanity.com/paper/2507.19477/) (2025)
- [Unified-MAS: Universally Generating Domain-Specific Nodes for Empowering Automatic Multi-Agent Systems](https://mlsanity.com/paper/2603.21475/) (2026)
- [PortBench: A Correlation-Aware, Full-Pipeline Benchmark for LLM-Driven Portfolio Management](https://mlsanity.com/paper/2605.27887/) (2026)
- [TradeTrap: Are LLM-based Trading Agents Truly Reliable and Faithful?](https://mlsanity.com/paper/2512.02261/) (2025)
- [LEAF: A Living Benchmark for Event-Augmented Forecasting](https://mlsanity.com/paper/2605.16358/) (2026)
- [From Natural Language to Executable Option Strategies via Large Language Models](https://mlsanity.com/paper/2603.16434/) (2026)
- [Can Agentic Trading Systems Pay for Their Own Intelligence?](https://mlsanity.com/paper/2607.10286/) (2026)
- [NextFund: A Unified Performance Tracking Platform for Agentic Portfolio Management](https://mlsanity.com/paper/2607.11141/) (2026)
- [EVOQUANT: Self-Evolving Verifier-Guided Strategy Optimization for Robust Quantitative Trading](https://mlsanity.com/paper/2607.12455/) (2026)
- [FinSAgent: Corpus-Aligned Multi-Agent RAG Framework for Evidence-Grounded SEC Filing Question Answering](https://mlsanity.com/paper/2607.18102/) (2026)
- [F2Agent: Financial Fusion of Agentic Intelligence for Multimodal Trading](https://mlsanity.com/paper/2608.05668/) (2026)
- [OpenPM: Auditable Point-in-Time Evaluation for LLM Portfolio-Management Agents](https://mlsanity.com/paper/2608.09988/) (2026)
- [Artificial Intelligence in Equity and Crypto Markets: Progress, Profitability Evidence, and the Limits of Automated Investing](https://mlsanity.com/paper/2609.04917/) (2026)
- [EverMine: Dissecting the Self-Evolution of Research Capabilities in Long-Horizon Alpha Research](https://mlsanity.com/paper/2609.33524/) (2026)
- [Can AI Make Money in Crypto? Measuring the Gap from Backtests to Real Markets](https://mlsanity.com/paper/2609.34510/) (2026)
- [From Search to Research: Exploring Search Scaling in Autonomous Quantitative Factor Mining](https://mlsanity.com/paper/2609.35559/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) have demonstrated notable capabilities across financial tasks, including financial report summarization, earnings call transcript analysis, and asset classification. However, their real-world effectiveness in managing complex fund investment remains inadequately assessed. A fundamental limitation of existing benchmarks for evaluating LLM-driven trading strategies is their reliance on historical back-testing, inadvertently enabling LLMs to "time travel"-leveraging future information embedded in their training corpora, thus resulting in possible information leakage and overly optimistic performance estimates. To address this issue, we introduce DeepFund, a live fund benchmark tool designed to rigorously evaluate LLM in real-time market conditions. Utilizing a multi-agent architecture, DeepFund connects directly with real-time stock market data-specifically data published after each model pretraining cutoff-to ensure fair and leakage-free evaluations. Empirical tests on nine flagship LLMs from leading global institutions across multiple investment dimensions-including ticker-level analysis, investment decision-making, portfolio management, and risk control-reveal significant practical challenges. Notably, even cutting-edge models such as DeepSeek-V3 and Claude-3.7-Sonnet incur net trading losses within DeepFund real-time evaluation environment, underscoring the present limitations of LLMs for active fund management. Our code is available at https://github.com/HKUSTDial/DeepFund.