mlsanityData to 7 Oct 2026

Frontier Financial Judgement: Can agents tell what might move a stock?

arXiv 2607.20645 ↗First posted 22 Jul 2026cs.CL cs.AI cs.LGcited by 0

On the map: LLM agents and tool use → Financial trading and forecasting agents

Abstract (arXiv)

We introduce Frontier Financial Judgement, a challenging new benchmark developed in collaboration with professional equity analysts to assess agents' ability to replicate expert human judgements. Rapidly identifying new information, evaluating its implications and determining its valuation impact is one of the most time-consuming and challenging aspects of real-world equity coverage. This is becoming ever more difficult and important as AI rapidly increases the quantity of new information to process. The strongest agent we evaluate on Frontier Financial Judgement matches all expert labels in only 52.4% of cases. We also find significant divergence in estimated false-positive rates among frontier agents, ranging from ~1% for GPT-5.6 Sol to ~32% for Claude Sonnet 4.6. To construct the benchmark and make it representative of real-world settings, we combine human-designed and labelled synthetic articles with live news articles and historical documents, creating 656 items for assessment. The resulting task requires agents to distinguish genuinely new, valuation-relevant financial information from stale, immaterial or misleading news under realistic conditions. We find substantial trade-offs among agent accuracy, cost, false positives and reliability that continue to hinder the reliable deployment of news-flow filtering in practice.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Financial trading and forecasting agents, one of 16 directions in LLM agents and tool use. Builds on 19 of its 35 references that are in the map. First posted on arXiv 22 Jul 2026.

Builds on

19 of 35 references.

Closest ancestors · 4

Same direction, newest first

  1. 2026BigFinanceBench: A Workflow-Grounded Benchmark for Financial-Research Agents2 Jun
  2. 2025FinSearchComp: Towards a Realistic, Expert-Level Evaluation of Financial Search and Reasoning16 Sep
  3. 2025FinAgentBench: A Benchmark Dataset for Agentic Retrieval in Financial Question Answering7 Aug
  4. 2025FinResearchBench: A Logic Tree based Agent-as-a-Judge Evaluation Framework for Financial Research Agents22 Jul

Other references · 15

  1. 2023FinanceBench: A New Benchmark for Financial Question Answeringcited by 261
  2. 2024LiveBench: A Challenging, Contamination-Limited LLM Benchmarkcited by 242
  3. 2023Stop Uploading Test Data in Plain Text: Practical Strategies for Mitigating Data Contamination by Evaluation Benchmarkscited by 139
  4. 2025Finance Agent Benchmark: Benchmarking LLMs on Real-world Financial Research Taskscited by 63
  5. 2021Trade the Event: Corporate Events Detection for News-Based Event-Driven Tradingcited by 39
  6. 2023Assessing Look-Ahead Bias in Stock Return Predictions Generated By GPT Sentiment Analysiscited by 29
  7. 2024FinDVer: Explainable Claim Verification over Long and Hybrid-Content Financial Documentscited by 19
  8. 2024EFSA: Towards Event-Level Financial Sentiment Analysiscited by 7
  9. 2025Towards Temporal-Aware Multi-Modal Retrieval Augmented Generation in Financecited by 7
  10. 2025ExAnte: A Benchmark for Ex-Ante Inference in Large Language Modelscited by 5
  11. 2026All That Glisters Is Not Gold: A Benchmark for Reference-Free Counterfactual Financial Misinformation Detectioncited by 4
  12. 2026AuditFraudBench: Benchmarking Audit Judgment in Detecting Fraudulent Misstatementscited by 3
3 more
  1. 2024FMDLlama: Financial Misinformation Detection based on Large Language Modelscited by 2
  2. 2026Point-in-Time Financial RAG with Frozen LLMs and Market-Feedback Adaptive Retrievalcited by 2
  3. 2025FinMarBa: A Market-Informed Dataset for Financial Sentiment Classificationcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026FrontierFinance: A Challenging Benchmark for Measuring Frontier Intelligence of Finance Agents
  2. 2026Hedge-Bench: Benchmarking Agents on Hard, Realistic Tasks Pertaining to Financial Reasoning
  3. 2025FinSphere, a Real-Time Stock Analysis Agent Powered by Instruction-Tuned LLMs and Domain Tools
  4. 2026FinRetrieval: A Benchmark for Financial Data Retrieval by AI Agents
  5. 2025FinDeepResearch: Evaluating Deep Research Agents in Rigorous Financial Analysis
  6. 2026From Knowing to Doing: A Memory-Controlled Benchmark for LLM Trading Agents on Stock Markets
  7. 2026Signal or Noise in Multi-Agent LLM-based Stock Recommendations?
  8. 2026FORCE-Bench: A Benchmark, Dataset, and Evaluation Harness for Agentic AI in Enterprise Finance