mlsanityData to 7 Oct 2026

DocFinQA: A Long-Context Financial Reasoning Dataset

arXiv 2401.06915 ↗First posted 12 Jan 2024cs.CL cs.AIcited by 59

On the map: LLM reasoning → Reasoning for agents, planning and code

Abstract (arXiv)

For large language models (LLMs) to be effective in the financial domain -- where each decision can have a significant impact -- it is necessary to investigate realistic tasks and data. Financial professionals often interact with documents that are hundreds of pages long, but most financial research datasets only deal with short excerpts from these documents. To address this, we introduce a long-document financial QA task. We augment 7,437 questions from the existing FinQA dataset with the full-document context, extending the average context length from under 700 words in FinQA to 123k words in DocFinQA. We conduct extensive experiments over retrieval-based QA pipelines and long-context language models. DocFinQA proves a significant challenge for even state-of-the-art systems. We also provide a case-study on the longest documents in DocFinQA and find that models particularly struggle on these documents. Addressing these challenges may have a wide reaching impact across applications where specificity and long-range contexts are critical, like gene sequences and legal document contract analysis.

Key facts. Cited by 59 papers in the map (data through 7 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 23 of its 41 references that are in the map. First posted on arXiv 12 Jan 2024.

Builds on

23 of 41 references.

References · 23

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  4. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  5. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  6. 2020Longformer: The Long-Document Transformercited by 3,362
  7. 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,160
  8. 2021Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolationcited by 1,085
  9. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 940
  10. 2023BloombergGPT: A Large Language Model for Financecited by 923
  11. 2023The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Onlycited by 755
  12. 2017The NarrativeQA Reading Comprehension Challengecited by 590
11 more
  1. 2021FinQA: A Dataset of Numerical Reasoning over Financial Datacited by 552
  2. 2021TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Financecited by 402
  3. 2017SearchQA: A New Q&A Dataset Augmented with Context from a Search Enginecited by 221
  4. 2023XuanYuan 2.0: A Large Chinese Financial Chat Model with Hundreds of Billions Parameterscited by 92
  5. 2023System 2 Attention (is something you might need too)cited by 85
  6. 2019Automated Rationale Generation: A Technique for Explainable AI and its Effects on Human Perceptionscited by 81
  7. 2023BBT-Fin: Comprehensive Construction of Chinese Financial Domain Pre-trained Language Model, Corpus and Benchmarkcited by 60
  8. 2020QED: A Framework and Dataset for Explanations in Question Answeringcited by 55
  9. 2022Extractive is not Faithful: An Investigation of Broad Unfaithfulness Problems in Extractive Summarizationcited by 32
  10. 2021Answer Generation for Retrieval-based Question Answering Systemscited by 13
  11. 2023A Graphical Approach to Document Layout Analysiscited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 9; 2025: 22; 2026: 28.

59 papers.

Sort:

By area:

  1. 2024BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystackcited by 211
  2. 2024TriForce: Lossless Acceleration of Long Sequence Generation with Hierarchical Speculative Decodingcited by 98
  3. 2024M-Longdoc: A Benchmark For Multimodal Super-Long Document Understanding And A Retrieval-Aware Tuning Frameworkcited by 36
  4. 2025Fino1: On the Transferability of Reasoning-Enhanced LLMs and Reinforcement Learning to Financecited by 32
  5. 2025FinAgentBench: A Benchmark Dataset for Agentic Retrieval in Financial Question Answeringcited by 29
  6. 2024Is It Really Long Context if All You Need Is Retrieval? Towards Genuinely Difficult Long Context NLPcited by 27
  7. 2025PeerQA: A Scientific Question Answering Dataset from Peer Reviewscited by 21
  8. 2025FinDER: Financial Dataset for Question Answering and Evaluating Retrieval-Augmented Generationcited by 17
  9. 2025PlanGEN: A Multi-Agent Framework for Generating Planning and Reasoning Trajectories for Complex Problem Solvingcited by 15
  10. 2024MathHay: An Automated Benchmark for Long-Context Mathematical Reasoning in LLMscited by 13
  11. 2025Emulating Retrieval Augmented Generation via Prompt Engineering for Enhanced Long Context Comprehension in LLMscited by 9
  12. 2025How Do Large Vision-Language Models See Text in Image? Unveiling the Distinctive Role of OCR Headscited by 9
38 more
  1. 2026Fin-RATE: A Real-world Financial Analytics and Tracking Evaluation Benchmark for LLMs on SEC Filingscited by 8
  2. 2024STRUX: An LLM for Decision-Making with Structured Explanationscited by 7
  3. 2025Single-Pass Document Scanning for Question Answeringcited by 6
  4. 2025LLM Output Drift: Cross-Provider Validation & Mitigation for Financial Workflowscited by 6
  5. 2026BigFinanceBench: A Workflow-Grounded Benchmark for Financial-Research Agentscited by 6
  6. 2025FinGEAR: Financial Mapping-Guided Enhanced Answer Retrievalcited by 5
  7. 2024Verbosity-Aware Rationale Reduction: Effective Reduction of Redundant Rationale via Principled Criteriacited by 4
  8. 2025Bridging Language Models and Financial Analysiscited by 4
  9. 2025Hierarchical Retrieval with Evidence Curation for Open-Domain Financial Question Answering on Standardized Documentscited by 4
  10. 2025Metadata-Driven Retrieval-Augmented Generation for Financial Question Answeringcited by 4
  11. 2026FinRetrieval: A Benchmark for Financial Data Retrieval by AI Agentscited by 4
  12. 2026FinGround: Detecting and Grounding Financial Hallucinations via Atomic Claim Verificationcited by 4
  13. 2024DeFine: Decision-Making with Analogical Reasoning over Factor Profilescited by 3
  14. 2025Extract, Match, and Score: An Evaluation Paradigm for Long Question-context-answer Triplets in Financial Analysiscited by 3
  15. 2025Language Modeling for the Future of Finance: A Survey into Metrics, Tasks, and Data Opportunitiescited by 3
  16. 2025VeritasFi: An Adaptable, Multi-tiered RAG Framework for Multi-modal Financial Question Answeringcited by 3
  17. 2026Herculean: An Agentic Benchmark for Financial Intelligencecited by 3
  18. 2026FinBalance: A Multi-Document Accounting Reconciliation Benchmarkcited by 3
  19. 2026FrontierFinance: A Challenging Benchmark for Measuring Frontier Intelligence of Finance Agentscited by 3
  20. 2025Domain Specific Benchmarks for Evaluating Multimodal Large Language Modelscited by 2
  21. 2026RoboPhD: Evolving Diverse Complex Agents Under Tight Evaluation Budgetscited by 2
  22. 2026FrontierFinance: A Long-Horizon Computer-Use Benchmark of Real-World Financial Taskscited by 2
  23. 2024Venn Diagram Prompting : Accelerating Comprehension with Scaffolding Effectcited by 1
  24. 2025FinReflectKG -- MultiHop: Financial QA Benchmark for Reasoning with Knowledge Graph Evidencecited by 1
  25. 2026A Decomposition Perspective to Long-context Reasoning for LLMscited by 1
  26. 2026FinRED: An Expert-Guided Benchmark Generation and Evaluation Framework for Financial LLM Red-Teamingcited by 1
  27. 2026INS-ActBench: A Comprehensive Benchmark for Assessing Professional Actuarial Capability of Large Language Modelscited by 1
  28. 2026FinDeepIndicator: Benchmarking Deep Research Agents in End-to-End Financial Indicator Constructioncited by 1
  29. 2025Fin-ExBERT: User Intent based Text Extraction in Financial Context using Graph-Augmented BERT and trainable Plugincited by 0
  30. 2025Query Generation Pipeline with Enhanced Answerability Assessment for Financial Information Retrievalcited by 0
  31. 2025Adaptation of Embedding Models to Financial Filings via LLM Distillationcited by 0
  32. 2025Financial Instruction Following Evaluation (FIFE)cited by 0
  33. 2026UniFinEval: Towards Unified Evaluation of Financial Multimodal Models across Text, Images and Videoscited by 0
  34. 2026NGDBench: Towards Neural Graph Data Managementcited by 0
  35. 2026π2: Structure-Originated Reasoning Data Improves Long-Context Reasoning Ability of Large Language Modelscited by 0
  36. 2026FinVerBench: Benchmark Validity and Calibration in Large Language Model Financial Statement Verificationcited by 0
  37. 2026MoCA-Agent: A Market-of-Claims Code Agent for Financial and Numerical Reasoningcited by 0
  38. 2026LEDGER: A Long-Context Benchmark of Corporate Annual Reports for Grounded Financial Retrieval and Extractioncited by 0

50 most cited of 59 All 59 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024SEC-QA: A Systematic Evaluation Corpus for Financial QA
  2. 2024FinTextQA: A Dataset for Long-form Financial Question Answering
  3. 2024FinLLMs: A Framework for Financial Reasoning Dataset Generation with Large Language Models
  4. 2026Document-Level Numerical Reasoning across Single and Multiple Tables in Financial Reports
  5. 2025FinLFQA: Evaluating Attributed Text Generation of LLMs in Financial Long-Form Question Answering
  6. 2023Beyond Classification: Financial Reasoning in State-of-the-Art Language Models
  7. 2025Evaluating Large Language Models for Financial Reasoning: A CFA-Based Benchmark Study
  8. 2026FinForge: Semi-Synthetic Financial Benchmark Generation