mlsanityData to 7 Oct 2026

Herculean: An Agentic Benchmark for Financial Intelligence

arXiv 2605.14355 ↗First posted 14 May 2026cs.AI cs.CLcited by 3

On the map: LLM agents and tool use → Financial trading and forecasting agents

Abstract (arXiv)

As AI agents improve, the central question is no longer whether they can solve isolated well-defined financial tasks, but whether they can reliably carry out financial professional work. Existing financial benchmarks offer only a partial view of this ability, as they primarily evaluate static competencies such as question answering, retrieval, summarization, and classification. We introduce Herculean, the first skilled benchmark for agentic financial intelligence spanning four representative workflows, including Trading, Hedging, Market Insights, and Auditing. Each workflow is instantiated as a standardized MCP-based skill environment with its own tools, interaction dynamics, constraints, and success criteria, enabling consistent end-to-end assessment of heterogeneous agent systems. Across frontier agents, we find agents perform relatively well on Trading and Market Insights, but struggle substantially on Hedging and Auditing, where long-horizon coordination, state consistency, and structured verification are critical. Overall, our results point to a key gap in current agents in turning financial reasoning into dependable workflow execution in high-stakes financial workflows.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Financial trading and forecasting agents, one of 16 directions in LLM agents and tool use. Builds on 23 of its 37 references that are in the map. First posted on arXiv 14 May 2026.

Builds on

23 of 37 references.

Closest ancestors · 4

Same direction, newest first

  1. 2026FinMCP-Bench: Benchmarking LLM Agents for Real-World Financial Tool Use under the Model Context Protocol26 Mar
  2. 2025StockBench: Can LLM Agents Trade Stocks Profitably In Real-world Markets?2 Oct
  3. 2025FinAgentBench: A Benchmark Dataset for Agentic Retrieval in Financial Question Answering7 Aug
  4. 2024INVESTORBENCH: A Benchmark for Financial Decision-Making Tasks with LLM-based Agent24 Dec

Other references · 19

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  2. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  3. 2023GAIA: a benchmark for General AI Assistantscited by 1,084
  4. 2021FinQA: A Dataset of Numerical Reasoning over Financial Datacited by 552
  5. 2023FinanceBench: A New Benchmark for Financial Question Answeringcited by 261
  6. 2024FinBen: A Holistic Financial Benchmark for Large Language Modelscited by 140
  7. 2024GTA: A Benchmark for General Tool Agentscited by 77
  8. 2025Finance Agent Benchmark: Benchmarking LLMs on Real-world Financial Research Taskscited by 63
  9. 2024DocFinQA: A Long-Context Financial Reasoning Datasetcited by 59
  10. 2025AstaBench: Rigorous Benchmarking of AI Agents with a Scientific Research Suitecited by 45
  11. 2026Benchmark Test-Time Scaling of General LLM Agentscited by 24
  12. 2025AI-Trader: Benchmarking Autonomous Agents in Real-Time Financial Marketscited by 23
7 more
  1. 2025FinChain: A Symbolic Benchmark for Verifiable Chain-of-Thought Financial Reasoningcited by 15
  2. 2025MultiFinBen: Benchmarking Large Language Models for Multilingual and Multimodal Financial Applicationcited by 14
  3. 2025FinAuditing: A Financial Taxonomy-Structured Multi-Document Benchmark for Evaluating LLMscited by 14
  4. 2025LiveTradeBench: Seeking Real-World Alpha with Large Language Modelscited by 13
  5. 2025XFinBench: Benchmarking LLMs in Complex Financial Problem Solving and Reasoningcited by 12
  6. 2025Finch: Benchmarking Finance & Accounting across Spreadsheet-Centric Enterprise Workflowscited by 9
  7. 2026Moira: Language-driven Hierarchical Reinforcement Learning for Pair Tradingcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026FinRetrieval: A Benchmark for Financial Data Retrieval by AI Agents
  2. 2026Mint-Agent: Introducing Finance-Native Agentic Foundation Models
  3. 2026Hedge-Bench: Benchmarking Agents on Hard, Realistic Tasks Pertaining to Financial Reasoning
  4. 2026FORCE-Bench: A Benchmark, Dataset, and Evaluation Harness for Agentic AI in Enterprise Finance
  5. 2026OpenFinGym: A Verifiable Multi-Task Gym Environment for Evaluating Quant Agents
  6. 2025VERAFI: Verified Agentic Financial Intelligence through Neurosymbolic Policy Generation
  7. 2024FISHNET: Financial Intelligence from Sub-querying, Harmonizing, Neural-Conditioning, Expert Swarms, and Task Planning
  8. 2024FinRobot: An Open-Source AI Agent Platform for Financial Applications using Large Language Models