# FinResearchBench: A Logic Tree based Agent-as-a-Judge Evaluation Framework for Financial Research Agents

arXiv: https://arxiv.org/abs/2507.16248 · HTML: https://mlsanity.com/paper/2507.16248/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Placed in Financial trading and forecasting agents, one of 16 directions in LLM agents and tool use.
- Builds on 17 of its 48 references that are in the map.
- First posted on arXiv 22 Jul 2025.

## Placed in

- LLM agents and tool use / [Financial trading and forecasting agents](https://mlsanity.com/agents/financial-trading-and-forecasting-agents/)

## Builds on (most cited first, 17 of 17)

- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context](https://mlsanity.com/paper/2403.05530/) (2024)
- [AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation](https://mlsanity.com/paper/2308.08155/) (2023)
- [MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework](https://mlsanity.com/paper/2308.00352/) (2023)
- [Mind2Web: Towards a Generalist Agent for the Web](https://mlsanity.com/paper/2306.06070/) (2023)
- [From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judge](https://mlsanity.com/paper/2411.16594/) (2024)
- [Agent Laboratory: Using LLM Agents as Research Assistants](https://mlsanity.com/paper/2501.04227/) (2025)
- [Contrastive Preference Optimization: Pushing the Boundaries of LLM Performance in Machine Translation](https://mlsanity.com/paper/2401.08417/) (2024)
- [ToolQA: A Dataset for LLM Question Answering with External Tools](https://mlsanity.com/paper/2306.13304/) (2023)
- [Sentiment Analysis in the Era of Large Language Models: A Reality Check](https://mlsanity.com/paper/2305.15005/) (2023)
- [A Survey on Recent Advances in LLM-Based Multi-turn Dialogue Systems](https://mlsanity.com/paper/2402.18013/) (2024)
- [LongWriter: Unleashing 10,000+ Word Generation from Long Context LLMs](https://mlsanity.com/paper/2408.07055/) (2024)
- [xbench: Tracking Agents Productivity Scaling with Profession-Aligned Real-World Evaluations](https://mlsanity.com/paper/2506.13651/) (2025)
- [LegalAgentBench: Evaluating LLM Agents in Legal Domain](https://mlsanity.com/paper/2412.17259/) (2024)
- [Deep Research Bench: Evaluating AI Web Research Agents](https://mlsanity.com/paper/2506.06287/) (2025)
- [LongEval: A Comprehensive Analysis of Long-Text Generation Through a Plan-based Paradigm](https://mlsanity.com/paper/2502.19103/) (2025)

## Built on by (most cited first, 12 of 12)

- [Agent-as-a-Judge](https://mlsanity.com/paper/2601.05111/) (2026)
- [FinDeepResearch: Evaluating Deep Research Agents in Rigorous Financial Analysis](https://mlsanity.com/paper/2510.13936/) (2025)
- [FinResearchBench II: A Deep Research Benchmark with Consensus-Derived Gold Rubrics for Distinguishing Financial Report Quality](https://mlsanity.com/paper/2607.12252/) (2026)
- [FrontierFinance: A Challenging Benchmark for Measuring Frontier Intelligence of Finance Agents](https://mlsanity.com/paper/2608.11683/) (2026)
- [Empowering Real-World: A Survey on the Technology, Practice, and Evaluation of LLM-driven Industry Agents](https://mlsanity.com/paper/2510.17491/) (2025)
- [Mind2Report: Expert-Level Commercial Report Synthesis via Cognitive Deep Research Agent](https://mlsanity.com/paper/2601.04879/) (2026)
- [JADE: Expert-Grounded Dynamic Evaluation for Open-Ended Professional Tasks](https://mlsanity.com/paper/2602.06486/) (2026)
- [Deep FinResearch Bench: Evaluating AI's Ability to Conduct Professional Financial Investment Research](https://mlsanity.com/paper/2604.21006/) (2026)
- [DFAH-Bench: Benchmarking Observable Agent Instability in Financial Decision-Making](https://mlsanity.com/paper/2607.20491/) (2026)
- [Frontier Financial Judgement: Can agents tell what might move a stock?](https://mlsanity.com/paper/2607.20645/) (2026)
- [From Simple QA to Deep Research: A Verifiable Benchmark Constructed through Iterative Task Evolution](https://mlsanity.com/paper/2608.02163/) (2026)
- [Agent Policy-Value Audit: Separating Transition Composition from Event Selection in Financial LLM Agents](https://mlsanity.com/paper/2610.04040/) (2026)

## Abstract (arXiv)

Recently, AI agents are rapidly evolving in intelligence and widely used in professional research applications, such as STEM, software development, and finance. Among these AI agents, deep research agent is a key category as it can perform long-horizon tasks and solve problems of greater complexity. However, there are few evaluation frameworks and benchmarks that systematically and automatically investigate the capabilities of these research agents. In addition, financial research problems have distinct complexity and subtlety. To fill in the gap, we propose FinResearchBench, which is a logic tree-based Agent-as-a-Judge and targets specifically for the financial research agents. It provides a comprehensive and automatic assessment of the research agents across 7 key types of tasks in the financial research domain. The contributions of this work are two-folded: (1) the first and innovative Agent-as-a-Judge system that extracts the logic tree of the research outcome and uses it as the intermediate information to present a comprehensive, reliable, and robust evaluation; (2) finance-oriented that it covers 70 typical financial research questions, spreading across 7 frequently encountered types of task in the domain.