mlsanityData to 7 Oct 2026

SECQUE: A Benchmark for Evaluating Real-World Financial Analysis Capabilities

arXiv 2504.04596 ↗First posted 6 Apr 2025cs.AI cs.CE cs.CLcited by 6

Not on the map. Why?

Abstract (arXiv)

We introduce SECQUE, a comprehensive benchmark for evaluating large language models (LLMs) in financial analysis tasks. SECQUE comprises 565 expert-written questions covering SEC filings analysis across four key categories: comparison analysis, ratio calculation, risk assessment, and financial insight generation. To assess model performance, we develop SECQUE-Judge, an evaluation mechanism leveraging multiple LLM-based judges, which demonstrates strong alignment with human evaluations. Additionally, we provide an extensive analysis of various models' performance on our benchmark. By making SECQUE publicly available, we aim to facilitate further research and advancements in financial AI.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 16 of its 27 references that are in the map. First posted on arXiv 6 Apr 2025.

Builds on

16 of 27 references.

References · 16

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  4. 2022Large Language Models Encode Clinical Knowledgecited by 1,807
  5. 2024A Survey on LLM-as-a-Judgecited by 1,384
  6. 2023BloombergGPT: A Large Language Model for Financecited by 923
  7. 2024Phi-4 Technical Reportcited by 675
  8. 2021FinQA: A Dataset of Numerical Reasoning over Financial Datacited by 552
  9. 2023Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyondcited by 409
  10. 2021TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Financecited by 402
  11. 2023FinanceBench: A New Benchmark for Financial Question Answeringcited by 261
  12. 2022ConvFinQA: Exploring the Chain of Numerical Reasoning in Conversational Finance Question Answeringcited by 194
4 more
  1. 2023BBT-Fin: Comprehensive Construction of Chinese Financial Domain Pre-trained Language Model, Corpus and Benchmarkcited by 60
  2. 2024Open-FinLLMs: Open Multimodal Large Language Models for Financial Applicationscited by 36
  3. 2024Mixing It Up: The Cocktail Effect of Multi-Task Fine-Tuning on LLM Performance -- A Case Study in Financecited by 15
  4. 2024SuperCLUE-Fin: Graded Fine-Grained Analysis of Chinese LLMs on Diverse Financial Tasks and Applicationscited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 6.

6 papers.

  1. 2026Fin-RATE: A Real-world Financial Analytics and Tracking Evaluation Benchmark for LLMs on SEC Filingscited by 8
  2. 2026FinSafetyBench: Evaluating LLM Safety in Real-World Financial Scenarioscited by 3
  3. 2026Utilizing Metadata for Better Retrieval-Augmented Generationcited by 1
  4. 2026FinEvo-Bench: A Longitudinal Benchmark for Self-Evolving Agents in Professional Financial Workflowscited by 1
  5. 2026FinVerBench: Benchmark Validity and Calibration in Large Language Model Financial Statement Verificationcited by 0
  6. 2026FINSKILLOPS: A Self-Evolving Multi-Agent System for SEC Filing QAcited by 0