mlsanityData to 7 Oct 2026

FinDVer: Explainable Claim Verification over Long and Hybrid-Content Financial Documents

arXiv 2411.05764 ↗First posted 8 Nov 2024cs.CL cs.LGcited by 19

Not on the map. Why?

Abstract (arXiv)

We introduce FinDVer, a comprehensive benchmark specifically designed to evaluate the explainable claim verification capabilities of LLMs in the context of understanding and analyzing long, hybrid-content financial documents. FinDVer contains 2,400 expert-annotated examples, divided into three subsets: information extraction, numerical reasoning, and knowledge-intensive reasoning, each addressing common scenarios encountered in real-world financial contexts. We assess a broad spectrum of LLMs under long-context and RAG settings. Our results show that even the current best-performing system, GPT-4o, still lags behind human experts. We further provide in-depth analysis on long-context and RAG setting, Chain-of-Thought reasoning, and model reasoning errors, offering insights to drive future advancements. We believe that FinDVer can serve as a valuable benchmark for evaluating LLMs in claim verification over complex, expert-domain documents.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Builds on 39 of its 64 references that are in the map. First posted on arXiv 8 Nov 2024.

Builds on

39 of 64 references.

References · 39

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  3. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  4. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,205
  5. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,582
  6. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,362
  7. 2021GLM: General Language Model Pretraining with Autoregressive Blank Infillingcited by 1,026
  8. 2024Yi: Open Foundation Models by 01.AIcited by 706
  9. 2021FinQA: A Dataset of Numerical Reasoning over Financial Datacited by 552
  10. 2020Fact or Fiction: Verifying Scientific Claimscited by 546
  11. 2019TabFact: A Large-scale Dataset for Table-based Fact Verificationcited by 425
  12. 2021TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Financecited by 402
27 more
  1. 2013Good Debt or Bad Debt: Detecting Semantic Orientations in Economic Textscited by 310
  2. 2024MiniCheck: Efficient Fact-Checking of LLMs on Grounding Documentscited by 251
  3. 2020HoVer: A Dataset for Many-Hop Fact Extraction And Claim Verificationcited by 199
  4. 2022ConvFinQA: Exploring the Chain of Numerical Reasoning in Conversational Finance Question Answeringcited by 194
  5. 2021FEVEROUS: Fact Extraction and VERification Over Unstructured and Structured informationcited by 166
  6. 2021ContractNLI: A Dataset for Document-level Natural Language Inference for Contractscited by 145
  7. 2023Stop Uploading Test Data in Plain Text: Practical Strategies for Mitigating Data Contamination by Evaluation Benchmarkscited by 139
  8. 2020Generating Fact Checking Explanationscited by 132
  9. 2022MultiHiertt: Numerical Reasoning over Multi Hierarchical Tabular and Textual Datacited by 120
  10. 2023WiCE: Real-World Entailment for Claims in Wikipediacited by 118
  11. 2022Large Language Models are few(1)-shot Table Reasonerscited by 116
  12. 2023Faithfulness Tests for Natural Language Explanationscited by 107
  13. 2022SciFact-Open: Towards open-domain scientific claim verificationcited by 94
  14. 2020INFOTABS: Inference on Tables as Semi-structured Datacited by 84
  15. 2021DocNLI: A Large-scale Dataset for Document-level Natural Language Inferencecited by 75
  16. 2022ECTSum: A New Benchmark Dataset For Bullet Point Summarization of Long Earnings Call Transcriptscited by 59
  17. 2023SCITAB: A Challenging Benchmark for Compositional Reasoning and Claim Verification on Scientific Tablescited by 51
  18. 2023RobuT: A Systematic Study of Table QA Robustness Against Human-Annotated Adversarial Perturbationscited by 41
  19. 2021Trade the Event: Corporate Events Detection for News-Based Event-Driven Tradingcited by 39
  20. 2023FinanceMath: Knowledge-Intensive Math Reasoning in Finance Domainscited by 39
  21. 2021AmbiFC: Fact-Checking Ambiguous Claims with Evidencecited by 38
  22. 2023EX-FEVER: A Dataset for Multi-hop Explainable Fact Verificationcited by 36
  23. 2024Open-FinLLMs: Open Multimodal Large Language Models for Financial Applicationscited by 36
  24. 2024Unveiling the Spectrum of Data Contamination in Language Models: A Survey from Detection to Remediationcited by 30
  25. 2023Benchmarking the Generation of Fact Checking Explanationscited by 25
  26. 2023Fin-Fact: A Benchmark Dataset for Multimodal Financial Fact Checking and Explanation Generationcited by 25
  27. 2023Long Text and Multi-Table Summarization: Dataset and Methodcited by 23

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 9; 2026: 10.

19 papers.

Sort:

By area:

  1. 2025SciVer: Evaluating Foundation Models for Multimodal Scientific Claim Verificationcited by 16
  2. 2025FinTrust: A Comprehensive Benchmark of Trustworthiness Evaluation in Finance Domaincited by 8
  3. 2026All That Glisters Is Not Gold: A Benchmark for Reference-Free Counterfactual Financial Misinformation Detectioncited by 4
  4. 2025Extract, Match, and Score: An Evaluation Paradigm for Long Question-context-answer Triplets in Financial Analysiscited by 3
  5. 2026ClaimDB: A Fact Verification Benchmark over Large Structured Datacited by 3
  6. 2025SUCEA: Reasoning-Intensive Retrieval for Adversarial Fact-checking through Claim Decomposition and Editingcited by 2
  7. 2025TSVer: A Benchmark for Fact Verification Against Time-Series Evidencecited by 2
  8. 2026Same Claim, Different Judgment: Benchmarking Scenario-Induced Bias in Multilingual Financial Misinformation Detectioncited by 2
  9. 2026Rethinking Reasoning-Intensive Retrieval: Evaluating and Advancing Retrievers in Agentic Search Systemscited by 2
  10. 2026Neurosymbolic Learning for Inference-Time Argumentationcited by 2
  11. 2025FinReflectKG -- MultiHop: Financial QA Benchmark for Reasoning with Knowledge Graph Evidencecited by 1
  12. 2025Towards Competent AI for Fundamental Analysis in Finance: A Benchmark Dataset and Evaluationcited by 0
7 more
  1. 2025FinLFQA: Evaluating Attributed Text Generation of LLMs in Financial Long-Form Question Answeringcited by 0
  2. 2025SynClaimEval: A Framework for Evaluating the Utility of Synthetic Data in Long-Context Claim Verificationcited by 0
  3. 2026Frame-Guided Synthetic Claim Generation for Automatic Fact-Checking Using High-Volume Tabular Datacited by 0
  4. 2026MEVER: Multi-Modal and Explainable Claim Verification with Graph-based Evidence Retrievalcited by 0
  5. 2026A Multi-Agent Approach for Claim Verification from Tabular Data Documentscited by 0
  6. 2026FinVerBench: Benchmark Validity and Calibration in Large Language Model Financial Statement Verificationcited by 0
  7. 2026Frontier Financial Judgement: Can agents tell what might move a stock?cited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026FinRule-Bench: A Benchmark for Joint Reasoning over Financial Tables and Principles
  2. 2026Are the Financial Reasoning from LLMs Credible? A Real World Test over Long-Horizon Statements
  3. 2026FinGround: Detecting and Grounding Financial Hallucinations via Atomic Claim Verification
  4. 2026V-FiLLM: Verified Financial LLM Reasoning Benchmark
  5. 2025FinanceReasoning: Benchmarking Financial Numerical Reasoning More Credible, Comprehensive and Challenging
  6. 2026FinRCA-Bench: Benchmarking Evidence Retrieval and Reasoning for Financial AI Systems
  7. 2023Beyond Classification: Financial Reasoning in State-of-the-Art Language Models
  8. 2025Evaluating Large Language Models for Financial Reasoning: A CFA-Based Benchmark Study