mlsanityData to 14 Oct 2026

SMARTFinRAG: Interactive Modularized Financial RAG Benchmark

arXiv 2504.18024 ↗First posted 25 Apr 2025cs.CE cs.CL cs.IRcited by 0

On the map: Retrieval-augmented generation → Legal and financial RAG

Abstract (arXiv)

Financial sectors are rapidly adopting language model technologies, yet evaluating specialized RAG systems in this domain remains challenging. This paper introduces SMARTFinRAG, addressing three critical gaps in financial RAG assessment: (1) a fully modular architecture where components can be dynamically interchanged during runtime; (2) a document-centric evaluation paradigm generating domain-specific QA pairs from newly ingested financial documents; and (3) an intuitive interface bridging research-implementation divides. Our evaluation quantifies both retrieval efficacy and response quality, revealing significant performance variations across configurations. The platform's open-source architecture supports transparent, reproducible research while addressing practical deployment challenges faced by financial institutions implementing RAG systems.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Legal and financial RAG, one of 15 directions in Retrieval-augmented generation. Builds on 6 of its 11 references that are in the map. First posted on arXiv 25 Apr 2025.

Builds on

6 of 11 references.

References · 6

  1. 2021FinQA: A Dataset of Numerical Reasoning over Financial Datacited by 562
  2. 2023Ragas: Automated Evaluation of Retrieval Augmented Generationcited by 418
  3. 2023FinanceBench: A New Benchmark for Financial Question Answeringcited by 270
  4. 2024FinBen: A Holistic Financial Benchmark for Large Language Modelscited by 141
  5. 2025FinArena: A Human-Agent Collaboration Framework for Financial Market Analysis and Forecastingcited by 8
  6. 2024FinQAPT: Empowering Financial Decisions with End-to-End LLM-driven Question Answering Pipelinecited by 6

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025FinS-Pilot: A Benchmark for Online Financial RAG System
  2. 2025VeritasFi: An Adaptable, Multi-tiered RAG Framework for Multi-modal Financial Question Answering
  3. 2024OmniEval: An Omnidirectional and Automatic RAG Evaluation Benchmark in Financial Domain
  4. 2026Bypassing Document Ingestion: An MCP Approach to Financial Q&A
  5. 2026Towards Expert Financial QA via Self-Improving RAG
  6. 2026MimirRAG: A Multi-Agent RAG Framework for Financial Data Retrieval with Metadata Integration
  7. 2025Rethinking Retrieval: From Traditional Retrieval Augmented Generation to Agentic and Non-Vector Reasoning Systems in the Financial Domain for Large Language Models
  8. 2024RAG-IT: Retrieval-Augmented Instruction Tuning for Automated Financial Analysis -- A Case Study for the Semiconductor Sector