# FinMCP-Bench: Benchmarking LLM Agents for Real-World Financial Tool Use under the Model Context Protocol

arXiv: https://arxiv.org/abs/2603.24943 · HTML: https://mlsanity.com/paper/2603.24943/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in Financial trading and forecasting agents, one of 16 directions in LLM agents and tool use.
- Builds on 13 of its 20 references that are in the map.
- First posted on arXiv 26 Mar 2026.

## Placed in

- LLM agents and tool use / [Financial trading and forecasting agents](https://mlsanity.com/agents/financial-trading-and-forecasting-agents/)

## Builds on (most cited first, 13 of 13)

- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [gpt-oss-120b & gpt-oss-20b Model Card](https://mlsanity.com/paper/2508.10925/) (2025)
- [ToolRL: Reward is All Tool Learning Needs](https://mlsanity.com/paper/2504.13958/) (2025)
- [ToRL: Scaling Tool-Integrated RL](https://mlsanity.com/paper/2503.23383/) (2025)
- [Tool-Star: Empowering LLM-Brained Multi-Tool Reasoner via Reinforcement Learning](https://mlsanity.com/paper/2505.16410/) (2025)
- [Synthetic Data Generation & Multi-Step RL for Reasoning & Tool Use](https://mlsanity.com/paper/2504.04736/) (2025)
- [FinanceReasoning: Benchmarking Financial Numerical Reasoning More Credible, Comprehensive and Challenging](https://mlsanity.com/paper/2506.05828/) (2025)
- [Benchmarking Large Language Models on CFLUE -- A Chinese Financial Language Understanding Evaluation Dataset](https://mlsanity.com/paper/2405.10542/) (2024)
- [CFBenchmark: Chinese Financial Assistant Benchmark for Large Language Model](https://mlsanity.com/paper/2311.05812/) (2023)
- [FinChain: A Symbolic Benchmark for Verifiable Chain-of-Thought Financial Reasoning](https://mlsanity.com/paper/2506.02515/) (2025)
- [CFinBench: A Comprehensive Chinese Financial Benchmark for Large Language Models](https://mlsanity.com/paper/2407.02301/) (2024)
- [Evaluating, Synthesizing, and Enhancing for Customer Support Conversation](https://mlsanity.com/paper/2508.04423/) (2025)
- [M3FinMeeting: A Multilingual, Multi-Sector, and Multi-Task Financial Meeting Understanding Evaluation Dataset](https://mlsanity.com/paper/2506.02510/) (2025)

## Built on by (most cited first, 8 of 8)

- [From Question Answering to Task Completion: A Survey on Agent System and Harness Design](https://mlsanity.com/paper/2606.20683/) (2026)
- [FinTrace: Holistic Trajectory-Level Evaluation of LLM Tool Calling for Long-Horizon Financial Tasks](https://mlsanity.com/paper/2604.10015/) (2026)
- [Herculean: An Agentic Benchmark for Financial Intelligence](https://mlsanity.com/paper/2605.14355/) (2026)
- [FinEvo-Bench: A Longitudinal Benchmark for Self-Evolving Agents in Professional Financial Workflows](https://mlsanity.com/paper/2608.06144/) (2026)
- [FinCUABuild: Can Agents Build Reliable Benchmarks for Dynamic Financial Computer Use?](https://mlsanity.com/paper/2609.07603/) (2026)
- [IndicBankBench: Evaluating Safety and Reliability of Language Model Assistants in Indian Retail Banking](https://mlsanity.com/paper/2609.29167/) (2026)
- [BazaarBench: Delegation Safety in Decentralized C2C Marketplaces Run by LLM Agents](https://mlsanity.com/paper/2610.06748/) (2026)
- [AlphaPilot: Multi-Turn Tool-Using LLM Agents for Adaptive Alpha Mining with Reinforcement Learning](https://mlsanity.com/paper/2610.15488/) (2026)

## Abstract (arXiv)

This paper introduces FinMCP-Bench, a novel benchmark for evaluating large language models (LLMs) in solving real-world financial problems through tool invocation of financial model context protocols. FinMCP-Bench contains 613 samples spanning 10 main scenarios and 33 sub-scenarios, featuring both real and synthetic user queries to ensure diversity and authenticity. It incorporates 65 real financial MCPs and three types of samples, single tool, multi-tool, and multi-turn, allowing evaluation of models across different levels of task complexity. Using this benchmark, we systematically assess a range of mainstream LLMs and propose metrics that explicitly measure tool invocation accuracy and reasoning capabilities. FinMCP-Bench provides a standardized, practical, and challenging testbed for advancing research on financial LLM agents.