mlsanityData to 7 Oct 2026

Language-Routed RAG and Direct Option Scoring for Multilingual Financial QA: DS@GT at FinMMEval

arXiv 2607.22841 ↗First posted 24 Jul 2026cs.IR cs.AI cs.CLcited by 0

On the map: Retrieval-augmented generation → Legal and financial RAG

Abstract (arXiv)

We present DS@GT's submission to FinMMEval 2026 Task 1, a multilingual financial exam question answering benchmark spanning English, Spanish, Greek, Chinese, and Hindi. Financial certification exams such as the CFA, EFPA, and CPA demand structured domain reasoning that standard NLP benchmarks do not capture, and this challenge compounds across languages where retrieval and representation infrastructure is underdeveloped. We build a retrieval-augmented pipeline on LangGraph that detects query language and retrieves semantically relevant exemplars from a 30,209-entry multilingual knowledge base using BGE-M3 embeddings and FAISS indexing. The system then scores answers via Retrieval-Augmented Direct Scoring (RADS), reading next-token log-probabilities over candidate option letters rather than generating free-form output. For low-resource languages, we fuse per-language and cross-lingual retrieval indices using weighted Reciprocal Rank Fusion. Model selection is language-routed: Qwen3-14B for Arabic, Chinese, and Hindi; Qwen2.5-14B for English; and Llama-3.1-8B for Greek, a routing derived from empirical ablations that reveal substantial language-asymmetric performance gaps. Notably, chain-of-thought prompting significantly degrades Greek accuracy (90.7% to 20.9%), and enabling Qwen3's default thinking mode collapses Arabic RADS performance to near-chance levels. Our results indicate that effective multilingual financial reasoning requires language-aware retrieval, model routing, and deliberate scoring strategy selection.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Legal and financial RAG, one of 15 directions in Retrieval-augmented generation. Builds on 12 of its 16 references that are in the map. First posted on arXiv 24 Jul 2026.

Builds on

12 of 16 references.

References · 12

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  5. 2017On Calibration of Modern Neural Networkscited by 4,860
  6. 2017Billion-scale similarity search with GPUscited by 2,477
  7. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,203
  8. 2024M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillationcited by 1,050
  9. 2023PIXIU: A Large Language Model, Instruction Data and Evaluation Benchmark for Financecited by 217
  10. 2024FinBen: A Holistic Financial Benchmark for Large Language Modelscited by 140
  11. 2023BizBench: A Quantitative Reasoning Benchmark for Business and Financecited by 28
  12. 2024CFinBench: A Comprehensive Chinese Financial Benchmark for Large Language Modelscited by 12

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Enhancing Financial Question Answering: A Novel Benchmark Dataset of Banks' financial statements
  2. 2025Evaluating Large Language Models for Financial Reasoning: A CFA-Based Benchmark Study
  3. 2024FinTextQA: A Dataset for Long-form Financial Question Answering
  4. 2026FIND: Toward Multimodal Financial Reasoning and Question Answering for Indic Languages
  5. 2025VeritasFi: An Adaptable, Multi-tiered RAG Framework for Multi-modal Financial Question Answering
  6. 2024FAMMA: A Benchmark for Financial Domain Multilingual Multimodal Question Answering
  7. 2025FinDER: Financial Dataset for Question Answering and Evaluating Retrieval-Augmented Generation
  8. 2025FinGEAR: Financial Mapping-Guided Enhanced Answer Retrieval