# StockBench: Can LLM Agents Trade Stocks Profitably In Real-world Markets?

arXiv: https://arxiv.org/abs/2510.02209 · HTML: https://mlsanity.com/paper/2510.02209/

## Key facts

- Cited by 44 papers in the map (data through 14 Oct 2026).
- Placed in Financial trading and forecasting agents, one of 16 directions in LLM agents and tool use.
- Builds on 26 of its 54 references that are in the map.
- First posted on arXiv 2 Oct 2025.

## Placed in

- LLM agents and tool use / [Financial trading and forecasting agents](https://mlsanity.com/agents/financial-trading-and-forecasting-agents/)

## Builds on (most cited first, 26 of 26)

- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://mlsanity.com/paper/2310.06770/) (2023)
- [DAPO: An Open-Source LLM Reinforcement Learning System at Scale](https://mlsanity.com/paper/2503.14476/) (2025)
- [SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering](https://mlsanity.com/paper/2405.15793/) (2024)
- [GAIA: a benchmark for General AI Assistants](https://mlsanity.com/paper/2311.12983/) (2023)
- [BloombergGPT: A Large Language Model for Finance](https://mlsanity.com/paper/2303.17564/) (2023)
- [FinQA: A Dataset of Numerical Reasoning over Financial Data](https://mlsanity.com/paper/2109.00122/) (2021)
- [GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Models](https://mlsanity.com/paper/2508.06471/) (2025)
- [TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Finance](https://mlsanity.com/paper/2105.07624/) (2021)
- [ConvFinQA: Exploring the Chain of Numerical Reasoning in Conversational Finance Question Answering](https://mlsanity.com/paper/2210.03849/) (2022)
- [xbench: Tracking Agents Productivity Scaling with Profession-Aligned Real-World Evaluations](https://mlsanity.com/paper/2506.13651/) (2025)
- [WHEN FLUE MEETS FLANG: Benchmarks and Large Pre-trained Language Model for Financial Domain](https://mlsanity.com/paper/2211.00083/) (2022)
- [A Survey of Large Language Models for Financial Applications: Progress, Prospects and Challenges](https://mlsanity.com/paper/2406.11903/) (2024)
- [A Survey of Large Language Models in Finance (FinLLMs)](https://mlsanity.com/paper/2402.02315/) (2024)
- [Deep Reinforcement Learning for Trading](https://mlsanity.com/paper/1911.10107/) (2019)
- [Finance Agent Benchmark: Benchmarking LLMs on Real-world Financial Research Tasks](https://mlsanity.com/paper/2508.00828/) (2025)
- [INVESTORBENCH: A Benchmark for Financial Decision-Making Tasks with LLM-based Agent](https://mlsanity.com/paper/2412.18174/) (2024)
- [When Thinking Fails: The Pitfalls of Reasoning for Instruction-Following in LLMs](https://mlsanity.com/paper/2505.11423/) (2025)
- [FinEval: A Chinese Financial Domain Knowledge Evaluation Benchmark for Large Language Models](https://mlsanity.com/paper/2308.09975/) (2023)
- [Scaling Reasoning, Losing Control: Evaluating Instruction Following in Large Reasoning Models](https://mlsanity.com/paper/2505.14810/) (2025)
- [FinSearchComp: Towards a Realistic, Expert-Level Evaluation of Financial Search and Reasoning](https://mlsanity.com/paper/2509.13160/) (2025)
- [BizFinBench: A Business-Driven Real-World Financial Benchmark for Evaluating LLMs](https://mlsanity.com/paper/2505.19457/) (2025)
- [FinPT: Financial Risk Prediction with Profile Tuning on Pretrained Foundation Models](https://mlsanity.com/paper/2308.00065/) (2023)
- [From Scores to Skills: A Cognitive Diagnosis Framework for Evaluating Financial Large Language Models](https://mlsanity.com/paper/2508.13491/) (2025)

## Built on by (most cited first, 30 of 44)

- [LiveTradeBench: Seeking Real-World Alpha with Large Language Models](https://mlsanity.com/paper/2511.03628/) (2025)
- [EcomBench: Towards Holistic Evaluation of Foundation Agents in E-commerce](https://mlsanity.com/paper/2512.08868/) (2025)
- [Agentic Environment Engineering for Large Language Models: A Survey of Environment Modeling, Synthesis, Evaluation, and Application](https://mlsanity.com/paper/2606.12191/) (2026)
- [Agentic Trading: When LLM Agents Meet Financial Markets](https://mlsanity.com/paper/2605.19337/) (2026)
- [Toward Reliable Evaluation of LLM-Based Financial Multi-Agent Systems: Taxonomy, Coordination Primacy, and Cost Awareness](https://mlsanity.com/paper/2603.27539/) (2026)
- [Knowing What You Know Is Not Enough: Large Language Model Confidences Don't Align With Their Actions](https://mlsanity.com/paper/2511.13240/) (2025)
- [Can LLM Agents Be CFOs? Benchmarking Long-Horizon Resource Allocation in an Uncertain Enterprise Environment](https://mlsanity.com/paper/2603.23638/) (2026)
- [Herculean: An Agentic Benchmark for Financial Intelligence](https://mlsanity.com/paper/2605.14355/) (2026)
- [KellyBench: A Benchmark for Long-Horizon Sequential Decision Making](https://mlsanity.com/paper/2604.27865/) (2026)
- [The Alpha Illusion: Reported Alpha from LLM Trading Agents Should Not Be Treated as Deployment Evidence](https://mlsanity.com/paper/2605.16895/) (2026)
- [FinEvolveBench: A Benchmark for Self-Evolving Agents on Low-Repetition Tasks with Implicit Rewards](https://mlsanity.com/paper/2606.06960/) (2026)
- [JADE: Expert-Grounded Dynamic Evaluation for Open-Ended Professional Tasks](https://mlsanity.com/paper/2602.06486/) (2026)
- [EcoGym: Evaluating LLMs for Long-Horizon Plan-and-Execute in Interactive Economies](https://mlsanity.com/paper/2602.09514/) (2026)
- [MALLES: A Multi-agent LLMs-based Economic Sandbox with Consumer Preference Alignment](https://mlsanity.com/paper/2603.17694/) (2026)
- [OpenFinGym: A Verifiable Multi-Task Gym Environment for Evaluating Quant Agents](https://mlsanity.com/paper/2606.26350/) (2026)
- [E-Commerce Bench: Evaluating LLM Agents on Long-Horizon Autonomous Business Operation](https://mlsanity.com/paper/2608.30730/) (2026)
- [PriceSeer: Evaluating Large Language Models in Real-Time Stock Prediction](https://mlsanity.com/paper/2601.06088/) (2025)
- [CN-Buzz2Portfolio: A Chinese-Market Dataset and Benchmark for LLM-Based Macro and Sector Asset Allocation from Daily Trending Financial News](https://mlsanity.com/paper/2603.22305/) (2026)
- [Learning to Trade Like an Expert: Cognitive Fine-Tuning for Stable Financial Reasoning in Language Models](https://mlsanity.com/paper/2604.16862/) (2026)
- [Strat-LLM: Stratified Strategy Alignment for LLM-based Stock Trading with Real-time Multi-Source Signals](https://mlsanity.com/paper/2605.06024/) (2026)
- [Paper Agents, Paper Gains: An Empirical Analysis of DeFi Investment Agents](https://mlsanity.com/paper/2605.29174/) (2026)
- [FinPersona-Bench: A Benchmark for Longitudinal Psychometric Stability of Autonomous Financial Agents](https://mlsanity.com/paper/2606.31522/) (2026)
- [ShiJianBench: From Dialogue to Decision for Long-Horizon Evaluation of Investment Advisors](https://mlsanity.com/paper/2608.01204/) (2026)
- [FM-Bench: A Benchmark for Long-Horizon Management with Competing Agents](https://mlsanity.com/paper/2608.18423/) (2026)
- [LLM Agents for Time-Series: A Survey](https://mlsanity.com/paper/2608.26226/) (2026)
- [LATTICE: Evaluating Decision Support Utility of Crypto Agents](https://mlsanity.com/paper/2604.26235/) (2026)
- [FinBoardBench: Benchmarking Dynamic Wealth Management and Strategic Financial Reasoning of LLMs via Board Game Simulations](https://mlsanity.com/paper/2605.27896/) (2026)
- [Probing Outcome-Level Resemblance and Mechanism-Level Alignment in LLM Risk Decisions: Evidence from the St. Petersburg Game](https://mlsanity.com/paper/2606.04978/) (2026)
- [Can Agentic Trading Systems Pay for Their Own Intelligence?](https://mlsanity.com/paper/2607.10286/) (2026)
- [NextFund: A Unified Performance Tracking Platform for Agentic Portfolio Management](https://mlsanity.com/paper/2607.11141/) (2026)

## Abstract (arXiv)

Large language models (LLMs) demonstrate strong potential as autonomous agents, with promising capabilities in reasoning, tool use, and sequential decision-making. While prior benchmarks have evaluated LLM agents in various domains, the financial domain remains underexplored, despite its significant economic value and complex reasoning requirements. Most existing financial benchmarks focus on static question-answering, failing to capture the dynamics of real-market trading. To address this gap, we introduce STOCKBENCH, a contamination-free benchmark designed to evaluate LLM agents in realistic, multi-month stock trading environments. Agents receive daily market signals -- including prices, fundamentals, and news -- and make sequential buy, sell, or hold decisions. Performance is measured using financial metrics such as cumulative return, maximum drawdown, and the Sortino ratio, capturing both profitability and risk management. We evaluate a wide range of state-of-the-art proprietary and open-source LLMs. Surprisingly, most models struggle to outperform the simple buy-and-hold baseline, while some models demonstrate the potential to achieve higher returns and stronger risk management. These findings highlight both the challenges and opportunities of LLM-based trading agents, showing that strong performance on static financial question-answering do not necessarily translate into effective trading behavior. We release STOCKBENCH as an open-source benchmark to enable future research on LLM-driven financial agents.