# FinTextQA: A Dataset for Long-form Financial Question Answering

arXiv: https://arxiv.org/abs/2405.09980 · HTML: https://mlsanity.com/paper/2405.09980/

## Key facts

- Cited by 54 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 54 references that are in the map.
- First posted on arXiv 16 May 2024.

## Builds on (most cited first, 27 of 27)

- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Qwen Technical Report](https://mlsanity.com/paper/2309.16609/) (2023)
- [Retrieval-Augmented Generation for Large Language Models: A Survey](https://mlsanity.com/paper/2312.10997/) (2023)
- [REALM: Retrieval-Augmented Language Model Pre-Training](https://mlsanity.com/paper/2002.08909/) (2020)
- [GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers](https://mlsanity.com/paper/2210.17323/) (2022)
- [MPNet: Masked and Permuted Pre-training for Language Understanding](https://mlsanity.com/paper/2004.09297/) (2020)
- [Towards General Text Embeddings with Multi-stage Contrastive Learning](https://mlsanity.com/paper/2308.03281/) (2023)
- [Baichuan 2: Open Large-scale Language Models](https://mlsanity.com/paper/2309.10305/) (2023)
- [ELI5: Long Form Question Answering](https://mlsanity.com/paper/1907.09190/) (2019)
- [Prometheus: Inducing Fine-grained Evaluation Capability in Language Models](https://mlsanity.com/paper/2310.08491/) (2023)
- [FinQA: A Dataset of Numerical Reasoning over Financial Data](https://mlsanity.com/paper/2109.00122/) (2021)
- [TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Finance](https://mlsanity.com/paper/2105.07624/) (2021)
- [FinanceBench: A New Benchmark for Financial Question Answering](https://mlsanity.com/paper/2311.11944/) (2023)
- [The Computational Limits of Deep Learning](https://mlsanity.com/paper/2007.05558/) (2020)
- [SOLAR 10.7B: Scaling Large Language Models with Simple yet Effective Depth Up-Scaling](https://mlsanity.com/paper/2312.15166/) (2023)
- [Lift Yourself Up: Retrieval-augmented Text Generation with Self Memory](https://mlsanity.com/paper/2305.02437/) (2023)
- [WebCPM: Interactive Web Search for Chinese Long-form Question Answering](https://mlsanity.com/paper/2305.06849/) (2023)
- [Qilin-Med-VL: Towards Chinese Large Vision-Language Model for General Healthcare](https://mlsanity.com/paper/2310.17956/) (2023)
- [AI-Generated Content (AIGC): A Survey](https://mlsanity.com/paper/2304.06632/) (2023)
- [On Pursuit of Designing Multi-modal Transformer for Video Grounding](https://mlsanity.com/paper/2109.06085/) (2021)
- [Evaluation Metrics in the Era of GPT-4: Reliably Evaluating Large Language Models on Sequence to Sequence Tasks](https://mlsanity.com/paper/2310.13800/) (2023)
- [Evaluating Instruction-Tuned Large Language Models on Code Comprehension and Generation](https://mlsanity.com/paper/2308.01240/) (2023)
- [LocVTP: Video-Text Pre-training for Temporal Localization](https://mlsanity.com/paper/2207.10362/) (2022)
- [Foundation Model-oriented Robustness: Robust Image Model Evaluation with Pretrained Models](https://mlsanity.com/paper/2308.10632/) (2023)
- [PreWoMe: Exploiting Presuppositions as Working Memory for Long Form Question Answering](https://mlsanity.com/paper/2310.16147/) (2023)

## Built on by (most cited first, 30 of 54)

- [HybridRAG: Integrating Knowledge Graphs and Vector Retrieval Augmented Generation for Efficient Information Extraction](https://mlsanity.com/paper/2408.04948/) (2024)
- [Time-MQA: Time Series Multi-Task Question Answering with Context Enhancement](https://mlsanity.com/paper/2503.01875/) (2025)
- [No Free Labels: Limitations of LLM-as-a-Judge Without Human Grounding](https://mlsanity.com/paper/2503.05061/) (2025)
- [FinMME: Benchmark Dataset for Financial Multi-Modal Reasoning Evaluation](https://mlsanity.com/paper/2505.24714/) (2025)
- [OmniEval: An Omnidirectional and Automatic RAG Evaluation Benchmark in Financial Domain](https://mlsanity.com/paper/2412.13018/) (2024)
- [Time Travel is Cheating: Going Live with DeepFund for Real-Time Fund Investment Benchmarking](https://mlsanity.com/paper/2505.11065/) (2025)
- [FinDER: Financial Dataset for Question Answering and Evaluating Retrieval-Augmented Generation](https://mlsanity.com/paper/2504.15800/) (2025)
- [FactCG: Enhancing Fact Checkers with Graph-Based Multi-Hop Data](https://mlsanity.com/paper/2501.17144/) (2025)
- [It's High Time: A Survey of Temporal Question Answering](https://mlsanity.com/paper/2505.20243/) (2025)
- [WixQA: A Multi-Dataset Benchmark for Enterprise Retrieval-Augmented Generation](https://mlsanity.com/paper/2505.08643/) (2025)
- [Towards Temporal-Aware Multi-Modal Retrieval Augmented Generation in Finance](https://mlsanity.com/paper/2503.05185/) (2025)
- [FinTrust: A Comprehensive Benchmark of Trustworthiness Evaluation in Finance Domain](https://mlsanity.com/paper/2510.15232/) (2025)
- [INS-MMBench: A Comprehensive Benchmark for Evaluating LVLMs' Performance in Insurance](https://mlsanity.com/paper/2406.09105/) (2024)
- [QuantEval: A Benchmark for Financial Quantitative Tasks in Large Language Models](https://mlsanity.com/paper/2601.08689/) (2026)
- [BigFinanceBench: A Workflow-Grounded Benchmark for Financial-Research Agents](https://mlsanity.com/paper/2606.03829/) (2026)
- [SustainableQA: A Comprehensive Question Answering Dataset for Corporate Sustainability and EU Taxonomy Reporting](https://mlsanity.com/paper/2508.03000/) (2025)
- [FinReporting: An Agentic Workflow for Localized Reporting of Cross-Jurisdiction Financial Disclosures](https://mlsanity.com/paper/2604.05966/) (2026)
- [Bridging Language Models and Financial Analysis](https://mlsanity.com/paper/2503.22693/) (2025)
- [Metadata-Driven Retrieval-Augmented Generation for Financial Question Answering](https://mlsanity.com/paper/2510.24402/) (2025)
- [FinRetrieval: A Benchmark for Financial Data Retrieval by AI Agents](https://mlsanity.com/paper/2603.04403/) (2026)
- [Extract, Match, and Score: An Evaluation Paradigm for Long Question-context-answer Triplets in Financial Analysis](https://mlsanity.com/paper/2503.16575/) (2025)
- [T2-RAGBench: Text-and-Table Benchmark for Evaluating Retrieval-Augmented Generation](https://mlsanity.com/paper/2506.12071/) (2025)
- [FinSafetyBench: Evaluating LLM Safety in Real-World Financial Scenarios](https://mlsanity.com/paper/2605.00706/) (2026)
- [FinSphere, a Real-Time Stock Analysis Agent Powered by Instruction-Tuned LLMs and Domain Tools](https://mlsanity.com/paper/2501.12399/) (2025)
- [LiveLongBench: Tackling Long-Context Understanding for Spoken Texts from Live Streams](https://mlsanity.com/paper/2504.17366/) (2025)
- [M3FinMeeting: A Multilingual, Multi-Sector, and Multi-Task Financial Meeting Understanding Evaluation Dataset](https://mlsanity.com/paper/2506.02510/) (2025)
- [How Accurate Are LLMs at Multi-Question Answering on Conversational Transcripts?](https://mlsanity.com/paper/2509.21732/) (2025)
- [KnowMT-Bench: Benchmarking Knowledge-Intensive Long-Form Question Answering in Multi-Turn Dialogues](https://mlsanity.com/paper/2509.21856/) (2025)
- [AlphaForgeBench: Benchmarking End-to-End Trading Strategy Design with Large Language Models](https://mlsanity.com/paper/2602.18481/) (2026)
- [Agentic Retrieval-Augmented Generation for Financial Document Question Answering](https://mlsanity.com/paper/2605.05409/) (2026)

## Abstract (arXiv)

Accurate evaluation of financial question answering (QA) systems necessitates a comprehensive dataset encompassing diverse question types and contexts. However, current financial QA datasets lack scope diversity and question complexity. This work introduces FinTextQA, a novel dataset for long-form question answering (LFQA) in finance. FinTextQA comprises 1,262 high-quality, source-attributed QA pairs extracted and selected from finance textbooks and government agency websites.Moreover, we developed a Retrieval-Augmented Generation (RAG)-based LFQA system, comprising an embedder, retriever, reranker, and generator. A multi-faceted evaluation approach, including human ranking, automatic metrics, and GPT-4 scoring, was employed to benchmark the performance of different LFQA system configurations under heightened noisy conditions. The results indicate that: (1) Among all compared generators, Baichuan2-7B competes closely with GPT-3.5-turbo in accuracy score; (2) The most effective system configuration on our dataset involved setting the embedder, retriever, reranker, and generator as Ada2, Automated Merged Retrieval, Bge-Reranker-Base, and Baichuan2-7B, respectively; (3) models are less susceptible to noise after the length of contexts reaching a specific threshold.