mlsanityData to 7 Oct 2026

INVESTORBENCH: A Benchmark for Financial Decision-Making Tasks with LLM-based Agent

arXiv 2412.18174 ↗First posted 24 Dec 2024cs.CE cs.AI q-fin.CPcited by 62

On the map: LLM agents and tool use → Financial trading and forecasting agents

Abstract (arXiv)

Recent advancements have underscored the potential of large language model (LLM)-based agents in financial decision-making. Despite this progress, the field currently encounters two main challenges: (1) the lack of a comprehensive LLM agent framework adaptable to a variety of financial tasks, and (2) the absence of standardized benchmarks and consistent datasets for assessing agent performance. To tackle these issues, we introduce textscInvestorBench, the first benchmark specifically designed for evaluating LLM-based agents in diverse financial decision-making contexts. InvestorBench enhances the versatility of LLM-enabled agents by providing a comprehensive suite of tasks applicable to different financial products, including single equities like stocks, cryptocurrencies and exchange-traded funds (ETFs). Additionally, we assess the reasoning and decision-making capabilities of our agent framework using thirteen different LLMs as backbone models, across various market environments and tasks. Furthermore, we have curated a diverse collection of open-source, multi-modal datasets and developed a comprehensive suite of environments for financial decision-making. This establishes a highly accessible platform for evaluating financial agents' performance across various scenarios.

Key facts. Cited by 62 papers in the map (data through 7 Oct 2026). Placed in Financial trading and forecasting agents, one of 16 directions in LLM agents and tool use. Builds on 18 of its 40 references that are in the map. First posted on arXiv 24 Dec 2024.

Builds on

18 of 40 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023FinMem: A Performance-Enhanced LLM Trading Agent with Layered Memory and Character Design23 Nov

Other references · 17

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  3. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
  4. 2023BloombergGPT: A Large Language Model for Financecited by 923
  5. 2024Yi: Open Foundation Models by 01.AIcited by 706
  6. 2019FinBERT: Financial Sentiment Analysis with Pre-trained Language Modelscited by 378
  7. 2023PIXIU: A Large Language Model, Instruction Data and Evaluation Benchmark for Financecited by 217
  8. 2020FinBERT: A Pretrained Language Model for Financial Communicationscited by 161
  9. 2024A Multimodal Foundation Agent for Financial Trading: Tool-Augmented, Diversified, and Generalistcited by 137
  10. 2022WHEN FLUE MEETS FLANG: Benchmarks and Large Pre-trained Language Model for Financial Domaincited by 117
  11. 2023InvestLM: A Large Language Model for Investment using Financial Domain Instruction Tuningcited by 80
  12. 2024Determinants of LLM-assisted Decision-Makingcited by 55
5 more
  1. 2021Trade the Event: Corporate Events Detection for News-Based Event-Driven Tradingcited by 39
  2. 2023Bayes in the age of intelligent machinescited by 22
  3. 2022Deep Reinforcement Learning Approach for Trading Automation in The Stock Marketcited by 9
  4. 2022FinBERT-MRC: financial named entity recognition using BERT under the machine reading comprehension paradigmcited by 8
  5. 2022Stock2Vec: An Embedding to Improve Predictive Models for Companiescited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 21; 2026: 41.

62 papers.

Sort:

By area:

  1. 2025Agentic Large Language Models, a surveycited by 105
  2. 2025StockBench: Can LLM Agents Trade Stocks Profitably In Real-world Markets?cited by 43
  3. 2026Graph-based Agent Memory: Taxonomy, Techniques, and Applicationscited by 22
  4. 2025Standard Benchmarks Fail -- Auditing LLM Agents in Finance Must Prioritize Riskcited by 13
  5. 2025LiveTradeBench: Seeking Real-World Alpha with Large Language Modelscited by 13
  6. 2025A Survey of Scaling in Large Language Model Reasoningcited by 10
  7. 2025From Deep Learning to LLMs: A survey of AI in Quantitative Investmentcited by 9
  8. 2025Finch: Benchmarking Finance & Accounting across Spreadsheet-Centric Enterprise Workflowscited by 9
  9. 2025Exposing Product Bias in LLM Investment Recommendationcited by 6
  10. 2025TextAtari: 100K Frames Game Playing with Language Agentscited by 6
  11. 2025FinGEAR: Financial Mapping-Guided Enhanced Answer Retrievalcited by 5
  12. 2025CryptoBench: A Dynamic Benchmark for Expert-Level Evaluation of LLM Agents in Cryptocurrencycited by 5
38 more
  1. 2026CLQT: A Closed-Loop, Cost-Aware, Strategy-Consistent Benchmark for Diagnostic Evaluation of LLM Portfolio-Management Agentscited by 5
  2. 2025Will LLMs be Professional at Fund Investment? DeepFund: A Live Arena Perspectivecited by 4
  3. 2025CFBenchmark-MM: Chinese Financial Assistant Benchmark for Multimodal Large Language Modelcited by 4
  4. 2026From Feedback Loops to Policy Updates: Reinforcement Fine-Tuning for LLM-Based Alpha Factor Discoverycited by 4
  5. 2026Conv-FinRe: A Conversational and Longitudinal Benchmark for Utility-Grounded Financial Recommendationcited by 3
  6. 2026Can LLM Agents Be CFOs? Benchmarking Long-Horizon Resource Allocation in an Uncertain Enterprise Environmentcited by 3
  7. 2026FinSafetyBench: Evaluating LLM Safety in Real-World Financial Scenarioscited by 3
  8. 2026Herculean: An Agentic Benchmark for Financial Intelligencecited by 3
  9. 2026PortBench: A Correlation-Aware, Full-Pipeline Benchmark for LLM-Driven Portfolio Managementcited by 3
  10. 2025FinLMM-R1: Enhancing Financial Reasoning in LMM through Scalable Data and Reward Designcited by 2
  11. 2025FinEval-KR: A Financial Domain Evaluation Framework for Large Language Models' Knowledge and Reasoningcited by 2
  12. 2025Evaluating Large Language Models (LLMs) in Financial NLP: A Comparative Study on Financial Report Analysiscited by 2
  13. 2025Empowering Real-World: A Survey on the Technology, Practice, and Evaluation of LLM-driven Industry Agentscited by 2
  14. 2026MALLES: A Multi-agent LLMs-based Economic Sandbox with Consumer Preference Alignmentcited by 2
  15. 2026FinTrace: Holistic Trajectory-Level Evaluation of LLM Tool Calling for Long-Horizon Financial Taskscited by 2
  16. 2026OpenFinGym: A Verifiable Multi-Task Gym Environment for Evaluating Quant Agentscited by 2
  17. 2025Trustworthy Reasoning: Evaluating and Enhancing Factual Accuracy in LLM Intermediate Thought Processescited by 1
  18. 2025TradeTrap: Are LLM-based Trading Agents Truly Reliable and Faithful?cited by 1
  19. 2025PriceSeer: Evaluating Large Language Models in Real-Time Stock Predictioncited by 1
  20. 2026TruthTensor: Evaluating LLMs through Human Imitation on Prediction Market under Drift and Holistic Reasoningcited by 1
  21. 2026CryptoAnalystBench: Failures in Multi-Tool Long-Form LLM Analysiscited by 1
  22. 2026Evaluating Financial Intelligence in Large Language Models: Benchmarking SuperInvesting AI with LLM Enginescited by 1
  23. 2026From Natural Language to Executable Option Strategies via Large Language Modelscited by 1
  24. 2026GAIN: A Benchmark for Goal-Aligned Decision-Making of Large Language Models under Imperfect Normscited by 1
  25. 2026TimeSeek: Temporal Reliability of Agentic Forecasterscited by 1
  26. 2026Context Training with Active Information Seekingcited by 1
  27. 2026INS-ActBench: A Comprehensive Benchmark for Assessing Professional Actuarial Capability of Large Language Modelscited by 1
  28. 2026ShiJianBench: From Dialogue to Decision for Long-Horizon Evaluation of Investment Advisorscited by 1
  29. 2026ERPBench: Evaluating LLM Agents for Enterprise Decision-Making Across Competitive Market Ecologiescited by 1
  30. 2025Benchmarking LLM Agents for Wealth-Management Workflowscited by 0
  31. 2026LATTICE: Evaluating Decision Support Utility of Crypto Agentscited by 0
  32. 2026Auditing demographic bias in AI-based emergency police dispatch: a cross-lingual evaluation of eleven large language modelscited by 0
  33. 2026DeepWeb-Bench: A Deep Research Benchmark Demanding Massive Cross-Source Evidence and Long-Horizon Derivationcited by 0
  34. 2026Probing Outcome-Level Resemblance and Mechanism-Level Alignment in LLM Risk Decisions: Evidence from the St. Petersburg Gamecited by 0
  35. 2026Beyond Agent Architecture: Execution Assumptions and Reproducibility in LLM-Based Trading Systemscited by 0
  36. 2026NextFund: A Unified Performance Tracking Platform for Agentic Portfolio Managementcited by 0
  37. 2026Frontier AI performance across the business disciplines: a case-grounded benchmark of knowledge work and analytical reasoningcited by 0
  38. 2026Capital Markets LLM Reliability Score (CM-LRS): From Plausible to Bankablecited by 0

50 most cited of 62 All 62 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025When Agents Trade: Live Multi-Market Trading Benchmark for LLM Agents
  2. 2026FinToolBench: Evaluating LLM Agents for Real-World Financial Tool Use
  3. 2024FinBen: A Holistic Financial Benchmark for Large Language Models
  4. 2024Large Language Model Agent in Financial Trading: A Survey
  5. 2026FinMCP-Bench: Benchmarking LLM Agents for Real-World Financial Tool Use under the Model Context Protocol
  6. 2026Evaluating Investment Logic in Large Language Models: A Real-World Benchmark Towards Personalzied Financial Agents
  7. 2026BizFinBench.v2: Towards Reliable LLMs in Finance via Real-User Data and Offline/Online Bilingual Evaluation
  8. 2026Toward Expert Investment Teams:A Multi-Agent LLM System with Fine-Grained Trading Tasks