Language-Routed RAG and Direct Option Scoring for Multilingual Financial QA: DS@GT at FinMMEval
On the map: Retrieval-augmented generation → Legal and financial RAG
Abstract (arXiv)
We present DS@GT's submission to FinMMEval 2026 Task 1, a multilingual financial exam question answering benchmark spanning English, Spanish, Greek, Chinese, and Hindi. Financial certification exams such as the CFA, EFPA, and CPA demand structured domain reasoning that standard NLP benchmarks do not capture, and this challenge compounds across languages where retrieval and representation infrastructure is underdeveloped. We build a retrieval-augmented pipeline on LangGraph that detects query language and retrieves semantically relevant exemplars from a 30,209-entry multilingual knowledge base using BGE-M3 embeddings and FAISS indexing. The system then scores answers via Retrieval-Augmented Direct Scoring (RADS), reading next-token log-probabilities over candidate option letters rather than generating free-form output. For low-resource languages, we fuse per-language and cross-lingual retrieval indices using weighted Reciprocal Rank Fusion. Model selection is language-routed: Qwen3-14B for Arabic, Chinese, and Hindi; Qwen2.5-14B for English; and Llama-3.1-8B for Greek, a routing derived from empirical ablations that reveal substantial language-asymmetric performance gaps. Notably, chain-of-thought prompting significantly degrades Greek accuracy (90.7% to 20.9%), and enabling Qwen3's default thinking mode collapses Arabic RADS performance to near-chance levels. Our results indicate that effective multilingual financial reasoning requires language-aware retrieval, model routing, and deliberate scoring strategy selection.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Legal and financial RAG, one of 15 directions in Retrieval-augmented generation. Builds on 12 of its 16 references that are in the map. First posted on arXiv 24 Jul 2026.
Builds on
12 of 16 references.
References · 12
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2024The Llama 3 Herd of Modelscited by 14,192
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
- 2020Measuring Massive Multitask Language Understandingcited by 7,768
- 2017On Calibration of Modern Neural Networkscited by 4,860
- 2017Billion-scale similarity search with GPUscited by 2,477
- 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,203
- 2024M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillationcited by 1,050
- 2023PIXIU: A Large Language Model, Instruction Data and Evaluation Benchmark for Financecited by 217
- 2024FinBen: A Holistic Financial Benchmark for Large Language Modelscited by 140
- 2023BizBench: A Quantitative Reasoning Benchmark for Business and Financecited by 28
- 2024CFinBench: A Comprehensive Chinese Financial Benchmark for Large Language Modelscited by 12
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Enhancing Financial Question Answering: A Novel Benchmark Dataset of Banks' financial statements
- 2025Evaluating Large Language Models for Financial Reasoning: A CFA-Based Benchmark Study
- 2024FinTextQA: A Dataset for Long-form Financial Question Answering
- 2026FIND: Toward Multimodal Financial Reasoning and Question Answering for Indic Languages
- 2025VeritasFi: An Adaptable, Multi-tiered RAG Framework for Multi-modal Financial Question Answering
- 2024FAMMA: A Benchmark for Financial Domain Multilingual Multimodal Question Answering
- 2025FinDER: Financial Dataset for Question Answering and Evaluating Retrieval-Augmented Generation
- 2025FinGEAR: Financial Mapping-Guided Enhanced Answer Retrieval