# AI-Trader: Benchmarking Autonomous Agents in Real-Time Financial Markets

arXiv: https://arxiv.org/abs/2512.10971 · HTML: https://mlsanity.com/paper/2512.10971/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Dec 2025.

## Built on by (most cited first, 25 of 25)

- [Autonomous Agents on Blockchains: Standards, Execution Models, and Trust Boundaries](https://mlsanity.com/paper/2601.04583/) (2026)
- [Agentic Trading: When LLM Agents Meet Financial Markets](https://mlsanity.com/paper/2605.19337/) (2026)
- [CLQT: A Closed-Loop, Cost-Aware, Strategy-Consistent Benchmark for Diagnostic Evaluation of LLM Portfolio-Management Agents](https://mlsanity.com/paper/2606.29771/) (2026)
- [Can LLM Agents Be CFOs? Benchmarking Long-Horizon Resource Allocation in an Uncertain Enterprise Environment](https://mlsanity.com/paper/2603.23638/) (2026)
- [Herculean: An Agentic Benchmark for Financial Intelligence](https://mlsanity.com/paper/2605.14355/) (2026)
- [Game-Theoretic Lens on LLM-based Multi-Agent Systems](https://mlsanity.com/paper/2601.15047/) (2026)
- [KellyBench: A Benchmark for Long-Horizon Sequential Decision Making](https://mlsanity.com/paper/2604.27865/) (2026)
- [When AI Fails, What Works? A Data-Driven Taxonomy of Real-World AI Risk Mitigation Strategies](https://mlsanity.com/paper/2603.04259/) (2026)
- [Reflections and New Directions for Human-Centered Large Language Models](https://mlsanity.com/paper/2605.06901/) (2026)
- [LEAF: A Living Benchmark for Event-Augmented Forecasting](https://mlsanity.com/paper/2605.16358/) (2026)
- [Aligning Language Models with Selective Prediction](https://mlsanity.com/paper/2607.03528/) (2026)
- [PriceSeer: Evaluating Large Language Models in Real-Time Stock Prediction](https://mlsanity.com/paper/2601.06088/) (2025)
- [Paper Agents, Paper Gains: An Empirical Analysis of DeFi Investment Agents](https://mlsanity.com/paper/2605.29174/) (2026)
- [EvolveTrade: Experience-Driven Policy Refinement for Self-Evolving LLM Trading Agents](https://mlsanity.com/paper/2609.17632/) (2026)
- [LATTICE: Evaluating Decision Support Utility of Crypto Agents](https://mlsanity.com/paper/2604.26235/) (2026)
- [FundaPod: A Multi-Persona Agent Pod Architecture with Knowledge Graph Memory for AI-Assisted Fundamental Investment Research](https://mlsanity.com/paper/2605.27864/) (2026)
- [Beyond Agent Architecture: Execution Assumptions and Reproducibility in LLM-Based Trading Systems](https://mlsanity.com/paper/2606.08285/) (2026)
- [Can Agentic Trading Systems Pay for Their Own Intelligence?](https://mlsanity.com/paper/2607.10286/) (2026)
- [NextFund: A Unified Performance Tracking Platform for Agentic Portfolio Management](https://mlsanity.com/paper/2607.11141/) (2026)
- [Evaluating Investment Logic in Large Language Models: A Real-World Benchmark Towards Personalzied Financial Agents](https://mlsanity.com/paper/2608.06108/) (2026)
- [LiveOption: Evaluating LLM Agents in Structured Option Trading with Nonlinear Payoffs](https://mlsanity.com/paper/2609.33470/) (2026)
- [PROACT-Agent: Progressive Runtime Oversight and Active Circuit-breaking for Real-Time Safety](https://mlsanity.com/paper/2609.34415/) (2026)
- [Can AI Make Money in Crypto? Measuring the Gap from Backtests to Real Markets](https://mlsanity.com/paper/2609.34510/) (2026)
- [SEER: Self-Evolving Event Reasoning and Retrieval for Time Series Forecasting](https://mlsanity.com/paper/2610.04109/) (2026)
- [LiveMACE: Process-Aware Evaluation of LLM Agent Capabilities in Evolving Markets](https://mlsanity.com/paper/2610.09872/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) have demonstrated remarkable potential as autonomous agents, approaching human-expert performance through advanced reasoning and tool orchestration. However, decision-making in fully dynamic and live environments remains highly challenging, requiring real-time information integration and adaptive responses. While existing efforts have explored live evaluation mechanisms in structured tasks, a critical gap remains in systematic benchmarking for real-world applications, particularly in finance where stringent requirements exist for live strategic responsiveness. To address this gap, we introduce AI-Trader, the first fully-automated, live, and data-uncontaminated evaluation benchmark for LLM agents in financial decision-making. AI-Trader spans three major financial markets: U.S. stocks, A-shares, and cryptocurrencies, with multiple trading granularities to simulate live financial environments. Our benchmark implements a revolutionary fully autonomous minimal information paradigm where agents receive only essential context and must independently search, verify, and synthesize live market information without human intervention. We evaluate six mainstream LLMs across three markets and multiple trading frequencies. Our analysis reveals striking findings: general intelligence does not automatically translate to effective trading capability, with most agents exhibiting poor returns and weak risk management. We demonstrate that risk control capability determines cross-market robustness, and that AI trading strategies achieve excess returns more readily in highly liquid markets than policy-driven environments. These findings expose critical limitations in current autonomous agents and provide clear directions for future improvements. The code and evaluation data are open-sourced to foster community research: https://github.com/HKUDS/AI-Trader.