mlsanityData to 7 Oct 2026

The CLEF-2026 FinMMEval Lab: Multilingual and Multimodal Evaluation of Financial AI Systems

arXiv 2602.10886 ↗First posted 11 Feb 2026cs.CL cs.AI cs.CEcited by 2

Not on the map. Why?

Abstract (arXiv)

We present the setup and the tasks of the FinMMEval Lab at CLEF 2026, which introduces the first multilingual and multimodal evaluation framework for financial Large Language Models (LLMs). While recent advances in financial natural language processing have enabled automated analysis of market reports, regulatory documents, and investor communications, existing benchmarks remain largely monolingual, text-only, and limited to narrow subtasks. FinMMEval 2026 addresses this gap by offering three interconnected tasks that span financial understanding, reasoning, and decision-making: Financial Exam Question Answering, Multilingual Financial Question Answering (PolyFiQA), and Financial Decision Making. Together, these tasks provide a comprehensive evaluation suite that measures models' ability to reason, generalize, and act across diverse languages and modalities. The lab aims to promote the development of robust, transparent, and globally inclusive financial AI systems, with datasets and evaluation resources publicly released to support reproducible research.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 27 of its 36 references that are in the map. First posted on arXiv 11 Feb 2026.

Builds on

27 of 36 references.

References · 27

  1. 2023BloombergGPT: A Large Language Model for Financecited by 923
  2. 2021FinQA: A Dataset of Numerical Reasoning over Financial Datacited by 552
  3. 2021TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Financecited by 402
  4. 2024R-Judge: Benchmarking Safety Risk Awareness for LLM Agentscited by 268
  5. 2023PIXIU: A Large Language Model, Instruction Data and Evaluation Benchmark for Financecited by 217
  6. 2022ConvFinQA: Exploring the Chain of Numerical Reasoning in Conversational Finance Question Answeringcited by 194
  7. 2024FinBen: A Holistic Financial Benchmark for Large Language Modelscited by 140
  8. 2022WHEN FLUE MEETS FLANG: Benchmarks and Large Pre-trained Language Model for Financial Domaincited by 117
  9. 2023FinGPT: Democratizing Internet-scale Data for Financial Large Language Modelscited by 65
  10. 2022ECTSum: A New Benchmark Dataset For Bullet Point Summarization of Long Earnings Call Transcriptscited by 59
  11. 2023The Next Chapter: A Study of Large Language Models in Storytellingcited by 58
  12. 2023FinanceMath: Knowledge-Intensive Math Reasoning in Finance Domainscited by 39
15 more
  1. 2024FIRE: Fact-checking with Iterative Retrieval and Verificationcited by 39
  2. 2020Financial Document Causality Detection Shared Task (FinCausal 2020)cited by 31
  3. 2023BizBench: A Quantitative Reasoning Benchmark for Business and Financecited by 28
  4. 2023Empowering Many, Biasing a Few: Generalist Credit Scoring through Large Language Modelscited by 20
  5. 2025FinMTEB: Finance Massive Text Embedding Benchmarkcited by 18
  6. 2023FiNER-ORD: Financial Named Entity Recognition Open Research Datasetcited by 17
  7. 2023REFinD: Relation Extraction Financial Datasetcited by 17
  8. 2025Plutus: Benchmarking Large Language Models in Low-Resource Greek Financecited by 16
  9. 2026FinCARDS: Card-Based Analyst Reranking for Financial Document Question Answeringcited by 16
  10. 2025FinChain: A Symbolic Benchmark for Verifiable Chain-of-Thought Financial Reasoningcited by 15
  11. 2024CFinBench: A Comprehensive Chinese Financial Benchmark for Large Language Modelscited by 12
  12. 2024AraFinNLP 2024: The First Arabic Financial NLP Shared Taskcited by 12
  13. 2026RealFin: How Well Do LLMs Reason About Finance When Users Leave Things Unsaid?cited by 9
  14. 2024D'olares or Dollars? Unraveling the Bilingual Prowess of Financial LLMs Between Spanish and Englishcited by 5
  15. 2024Can AI Help with Your Personal Finances?cited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.