FinDeepForecast: A Live Multi-Agent System for Benchmarking Deep Research Agents in Financial Forecasting
On the map: LLM agents and tool use → Financial trading and forecasting agents
Abstract (arXiv)
Deep Research (DR) Agents powered by advanced Large Language Models (LLMs) have fundamentally shifted the paradigm for completing complex research tasks. Yet, a comprehensive and live evaluation of their forecasting performance on real-world, research-oriented tasks in high-stakes domains (e.g., finance) remains underexplored. We introduce FinDeepForecast, the first live, end-to-end multi-agent system for automatically evaluating DR agents by continuously generating research-oriented financial forecasting tasks. This system is equipped with a dual-track taxonomy, enabling the dynamic generation of recurrent and non-recurrent forecasting tasks at both corporate and macro levels. With this system, we generate FinDeepForecastBench, a weekly evaluation benchmark over a ten-week horizon, encompassing 8 global economies and 1,314 listed companies, and evaluate 13 representative methods. Extensive experiments show that, while DR agents consistently outperform strong baselines, their performance still falls short of genuine forward-looking financial reasoning. We expect the proposed FinDeepForecast system to consistently facilitate future advancements of DR agents in research-oriented financial forecasting tasks. The benchmark and leaderboard are publicly available on the OpenFinArena Platform.
Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Financial trading and forecasting agents, one of 16 directions in LLM agents and tool use. Builds on 18 of its 33 references that are in the map. First posted on arXiv 8 Jan 2026.
Builds on
18 of 33 references.
Closest ancestors · 2
Same direction, newest first
- 2025FinDeepResearch: Evaluating Deep Research Agents in Rigorous Financial Analysis15 Oct
- 2025FutureX: An Advanced Live Benchmark for LLM Agents in Future Prediction16 Aug
Other references · 16
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
- 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 1,908
- 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 1,906
- 2023NLP Evaluation in trouble: On the Need to Measure LLM Data Contamination for each Benchmarkcited by 339
- 2025MathArena: Evaluating LLMs on Uncontaminated Math Competitionscited by 275
- 2023FinanceBench: A New Benchmark for Financial Question Answeringcited by 261
- 2024LiveBench: A Challenging, Contamination-Limited LLM Benchmarkcited by 242
- 2024Leak, Cheat, Repeat: Data Contamination and Evaluation Malpractices in Closed-Source LLMscited by 224
- 2023PIXIU: A Large Language Model, Instruction Data and Evaluation Benchmark for Financecited by 217
- 2025Deep Research Agents: A Systematic Examination And Roadmapcited by 122
- 2022WHEN FLUE MEETS FLANG: Benchmarks and Large Pre-trained Language Model for Financial Domaincited by 117
4 more
- 2024ForecastBench: A Dynamic Benchmark of AI Forecasting Capabilitiescited by 104
- 2017Listening to Chaotic Whispers: A Deep Learning Framework for News-oriented Stock Trend Predictioncited by 60
- 2025CAMEF: Causal-Augmented Multi-Modality Event-Driven Financial Forecasting by Integrating Time Series Patterns and Salient Macroeconomic Announcementscited by 5
- 2025FinCall-Surprise: A Large Scale Multi-modal Benchmark for Earning Surprise Predictioncited by 2
Built on by
Citing papers per year: 2026: 4.
4 papers.
- 2026FinTrace: Holistic Trajectory-Level Evaluation of LLM Tool Calling for Long-Horizon Financial Taskscited by 2
- 2026FinDeepIndicator: Benchmarking Deep Research Agents in End-to-End Financial Indicator Constructioncited by 1
- 2026DelistBench: Evaluating Search-Enabled LLMs for Auditable Corporate-Event Database Completioncited by 0
- 2026LLM-based Agents for Forecasting and Prediction: Methods, Training, Evaluation, and Applicationscited by 0
- 2026LLM-based Agents for Forecasting and Prediction: Methods, Training, Evaluation, and Applications24 Aug 2026
- 2026DelistBench: Evaluating Search-Enabled LLMs for Auditable Corporate-Event Database Completion24 Aug 2026
- 2026FinDeepIndicator: Benchmarking Deep Research Agents in End-to-End Financial Indicator Construction1 Aug 2026
- 2026FinTrace: Holistic Trajectory-Level Evaluation of LLM Tool Calling for Long-Horizon Financial Tasks11 Apr 2026
Similar, not cited
Close in wording, no citation either way
- 2025FinResearchBench: A Logic Tree based Agent-as-a-Judge Evaluation Framework for Financial Research Agents
- 2025MASFIN: A Multi-Agent System for Decomposed Financial Reasoning and Forecasting
- 2026FinanceHarness: Autonomous Financial Deep Research Framework
- 2026Time Series Augmented Generation for Financial Applications
- 2026OpenFinGym: A Verifiable Multi-Task Gym Environment for Evaluating Quant Agents
- 2026Mint-Agent: Introducing Finance-Native Agentic Foundation Models
- 2026EvoCast: Reliable Autonomous Research Agents for Iterative Forecasting Architecture Evolution
- 2026From Narrative to Auditable Forecasts: A Structured Scaffold for Agentic Forecasting