# FinMTM: A Multi-Turn Multimodal Benchmark for Financial Reasoning and Agent Evaluation

arXiv: https://arxiv.org/abs/2602.03130 · HTML: https://mlsanity.com/paper/2602.03130/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning.
- Builds on 15 of its 33 references that are in the map.
- First posted on arXiv 3 Feb 2026.

## Placed in

- LLM reasoning / [Reasoning for agents, planning and code](https://mlsanity.com/lr/reasoning-for-agents-planning-and-code/)
- Multimodal LLMs / vision-language models / [Chart understanding and anomaly detection](https://mlsanity.com/mllm/chart-understanding-and-anomaly-detection/)
- LLM agents and tool use / [Financial trading and forecasting agents](https://mlsanity.com/agents/financial-trading-and-forecasting-agents/)

## Builds on (most cited first, 15 of 15)

- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models](https://mlsanity.com/paper/2504.10479/) (2025)
- [Mixtral of Experts](https://mlsanity.com/paper/2401.04088/) (2024)
- [Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling](https://mlsanity.com/paper/2412.05271/) (2024)
- [BloombergGPT: A Large Language Model for Finance](https://mlsanity.com/paper/2303.17564/) (2023)
- [GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learning](https://mlsanity.com/paper/2507.01006/) (2025)
- [MiMo: Unlocking the Reasoning Potential of Language Model -- From Pretraining to Posttraining](https://mlsanity.com/paper/2505.07608/) (2025)
- [R&D-Agent-Quant: A Multi-Agent Framework for Data-Centric Factors and Model Joint Optimization](https://mlsanity.com/paper/2505.15155/) (2025)
- [FinMME: Benchmark Dataset for Financial Multi-Modal Reasoning Evaluation](https://mlsanity.com/paper/2505.24714/) (2025)
- [FinDeepResearch: Evaluating Deep Research Agents in Rigorous Financial Analysis](https://mlsanity.com/paper/2510.13936/) (2025)
- [Towards Temporal-Aware Multi-Modal Retrieval Augmented Generation in Finance](https://mlsanity.com/paper/2503.05185/) (2025)
- [CFBenchmark-MM: Chinese Financial Assistant Benchmark for Multimodal Large Language Model](https://mlsanity.com/paper/2506.13055/) (2025)
- [FinCriticalED: A Visual Benchmark for Financial Fact-Level OCR](https://mlsanity.com/paper/2511.14998/) (2025)
- [Innovative use of X-ray radiography in the study of daguerreotypes: identification of hallmarks](https://mlsanity.com/paper/2307.00001/) (2023)
- [RETuning: Upgrading Inference-Time Scaling for Stock Movement Prediction with Large Language Models](https://mlsanity.com/paper/2510.21604/) (2025)

## Built on by (most cited first, 4 of 4)

- [Agentic Environment Engineering for Large Language Models: A Survey of Environment Modeling, Synthesis, Evaluation, and Application](https://mlsanity.com/paper/2606.12191/) (2026)
- [TimeSage-MT: A Multi-Turn Benchmark for Evaluating Agentic Time Series Reasoning](https://mlsanity.com/paper/2606.01498/) (2026)
- [Messier: A High-Resolution Corpus for Cross-Benchmark Agent Evaluation](https://mlsanity.com/paper/2607.25891/) (2026)
- [BazaarBench: Delegation Safety in Decentralized C2C Marketplaces Run by LLM Agents](https://mlsanity.com/paper/2610.06748/) (2026)

## Abstract (arXiv)

The financial domain poses substantial challenges for vision-language models (VLMs) due to specialized chart formats and knowledge-intensive reasoning requirements. However, existing financial benchmarks are largely single-turn and rely on a narrow set of question formats, limiting comprehensive evaluation in realistic application scenarios. To address this gap, we propose FinMTM, a multi-turn multimodal benchmark that expands diversity along both data and task dimensions. On the data side, we curate and annotate 11{,}133 bilingual (Chinese and English) financial QA pairs grounded in financial visuals, including candlestick charts, statistical plots, and report figures. On the task side, FinMTM covers single- and multiple-choice questions, multi-turn open-ended dialogues, and agent-based tasks. We further design task-specific evaluation protocols, including a set-overlap scoring rule for multiple-choice questions, a weighted combination of turn-level and session-level scores for multi-turn dialogues, and a composite metric that integrates planning quality with final outcomes for agent tasks. Extensive experimental evaluation of 22 VLMs reveal their limitations in fine-grained visual perception, long-context reasoning, and complex agent workflows.