mlsanityData to 14 Oct 2026

FinReportBench: Measuring and Improving Institution-Grade Financial Report Generation

arXiv 2608.04374 ↗First posted 5 Aug 2026cs.CL cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Large language models can produce fluent financial analysis, but fluency alone does not establish whether a report is suitable for institutional delivery. We introduce FinReportBench, an expert-grounded benchmark for measuring and improving institution-grade financial report generation. Expert review reveals recurring gaps in report identity, institutional components, source discipline, and visual delivery. We derive a 35-item rubric through expert partial orders, multimodal evidence, and audits of decision boundaries, covering deliverability, report identity, and institutional completeness. Starting from 10,000 balanced Chinese and English financial-research source records, we curate 244 bilingual tasks across three research objects and two input tiers. Each task separates the public query, reconstructed research trajectory, and hidden source packet. Three independent judge families reproduce the expert partial order at near-ceiling rates, showing that bounded, observable criteria support reliable evaluation. Across nine model families, basic deliverability is nearly saturated, while report identity and institutional completeness remain the primary bottlenecks. The largest cross-model gaps concern generation-trace control, information density, and data discipline rather than basic report framing. We then use benchmark-guided skill distillation to turn recurrent failures into reusable generation and self-review constraints. Across five model families, the evolved skill improves mean G1 by 33.85 points and mean G2 by 13.83 points over paired no-skill runs while preserving G0 for every pair. Code and benchmark artifacts are available at https://github.com/MisterBrookT/finreportbench.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 38 of its 45 references that are in the map. First posted on arXiv 5 Aug 2026.

Builds on

38 of 45 references.

References · 38

  1. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  2. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 2,006
  3. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,745
  4. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,479
  5. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,444
  6. 2023Mind2Web: Towards a Generalist Agent for the Webcited by 1,242
  7. 2024Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preferencecited by 1,184
  8. 2023GAIA: a benchmark for General AI Assistantscited by 1,128
  9. 2023BloombergGPT: A Large Language Model for Financecited by 939
  10. 2021FinQA: A Dataset of Numerical Reasoning over Financial Datacited by 562
  11. 2021TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Financecited by 408
  12. 2019PlotQA: Reasoning over Scientific Plotscited by 321
26 more
  1. 2024JudgeBench: A Benchmark for Evaluating LLM-based Judgescited by 318
  2. 2026SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Taskscited by 315
  3. 2023FinanceBench: A New Benchmark for Financial Question Answeringcited by 270
  4. 2024Agent-as-a-Judge: Evaluate Agents with Agentscited by 238
  5. 2023PIXIU: A Large Language Model, Instruction Data and Evaluation Benchmark for Financecited by 222
  6. 2022ConvFinQA: Exploring the Chain of Numerical Reasoning in Conversational Finance Question Answeringcited by 196
  7. 2025WebDancer: Towards Autonomous Information Seeking Agencycited by 188
  8. 2023UniChart: A Universal Vision-language Pretrained Model for Chart Comprehension and Reasoningcited by 160
  9. 2026CoEvoSkills: Self-Evolving Agent Skills via Co-Evolutionary Verificationcited by 128
  10. 2022MultiHiertt: Numerical Reasoning over Multi Hierarchical Tabular and Textual Datacited by 121
  11. 2025Mind2Web 2: Evaluating Agentic Search with Agent-as-a-Judgecited by 76
  12. 2022OpenCQA: Open-ended Question Answering with Chartscited by 61
  13. 2024DocFinQA: A Long-Context Financial Reasoning Datasetcited by 60
  14. 2024LongGenBench: Benchmarking Long-Form Generation in Long Context LLMscited by 47
  15. 2026SKILLFOUNDRY: Building Self-Evolving Agent Skill Libraries from Heterogeneous Scientific Resourcescited by 41
  16. 2025FinMME: Benchmark Dataset for Financial Multi-Modal Reasoning Evaluationcited by 30
  17. 2026From Raw Experience to Skill Consumption: A Systematic Study of Model-Generated Agent Skillscited by 22
  18. 2025VisJudge-Bench: Aesthetics and Quality Assessment of Visualizationscited by 16
  19. 2025You Don't Know Until You Click:Automated GUI Testing for Production-Ready Software Evaluationcited by 12
  20. 2025FinSight: Towards Real-World Financial Deep Researchcited by 9
  21. 2024DataLab: A Unified Platform for LLM-Powered Business Intelligencecited by 8
  22. 2026DeepEye: A Steerable Self-driving Data Agent Systemcited by 8
  23. 2025ChartMark: A Structured Grammar for Chart Annotationcited by 6
  24. 2026Demonstrating chart-plot: Closing the Last Mile of Academic Chart Generationcited by 4
  25. 2026ViviDoc: Generating Interactive Documents through Human-Agent Collaborationcited by 2
  26. 2026DataMagic: Transforming Tabular Data into Data Insight Videocited by 2

Built on by

Not cited yet.