mlsanityData to 14 Oct 2026

FinTrust: A Comprehensive Benchmark of Trustworthiness Evaluation in Finance Domain

arXiv 2510.15232 ↗First posted 17 Oct 2025cs.LG cs.CLcited by 8

Not on the map. Why?

Abstract (arXiv)

Recent LLMs have demonstrated promising ability in solving finance related problems. However, applying LLMs in real-world finance application remains challenging due to its high risk and high stakes property. This paper introduces FinTrust, a comprehensive benchmark specifically designed for evaluating the trustworthiness of LLMs in finance applications. Our benchmark focuses on a wide range of alignment issues based on practical context and features fine-grained tasks for each dimension of trustworthiness evaluation. We assess eleven LLMs on FinTrust and find that proprietary models like o4-mini outperforms in most tasks such as safety while open-source models like DeepSeek-V3 have advantage in specific areas like industry-level fairness. For challenging task like fiduciary alignment and disclosure, all LLMs fall short, showing a significant gap in legal awareness. We believe that FinTrust can be a valuable benchmark for LLMs' trustworthiness evaluation in finance domain.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 29 of its 49 references that are in the map. First posted on arXiv 17 Oct 2025.

Builds on

29 of 49 references.

References · 29

  1. 2021FinQA: A Dataset of Numerical Reasoning over Financial Datacited by 562
  2. 2023Large Language Models Are Zero-Shot Time Series Forecasterscited by 500
  3. 2024Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attackscited by 486
  4. 2021TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Financecited by 408
  5. 2023Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignmentcited by 401
  6. 2024Justice or Prejudice? Quantifying Biases in LLM-as-a-Judgecited by 368
  7. 2023DeepInception: Hypnotize Large Language Model to Be Jailbreakercited by 367
  8. 2021Societal Biases in Language Generation: Progress and Challengescited by 202
  9. 2024Don't Hallucinate, Abstain: Identifying LLM Knowledge Gaps via Multi-LLM Collaborationcited by 180
  10. 2022MultiHiertt: Numerical Reasoning over Multi Hierarchical Tabular and Textual Datacited by 121
  11. 2024Relying on the Unreliable: The Impact of Language Models' Reluctance to Express Uncertaintycited by 106
  12. 2024The Language Barrier: Dissecting Safety Challenges of LLMs in Multilingual Contextscited by 94
17 more
  1. 2024CheckEval: A reliable LLM-as-a-Judge framework for evaluating text generation using checklistscited by 94
  2. 2023XuanYuan 2.0: A Large Chinese Financial Chat Model with Hundreds of Billions Parameterscited by 92
  3. 2021AI in Finance: Challenges, Techniques and Opportunitiescited by 81
  4. 2021Fairness in Credit Scoring: Assessment, Implementation and Profit Implicationscited by 65
  5. 2024FinTextQA: A Dataset for Long-form Financial Question Answeringcited by 54
  6. 2023DocMath-Eval: Evaluating Math Reasoning Capabilities of LLMs in Understanding Long and Specialized Documentscited by 45
  7. 2023Deficiency of Large Language Models in Finance: An Empirical Examination of Hallucinationcited by 41
  8. 2023FinanceMath: Knowledge-Intensive Math Reasoning in Finance Domainscited by 40
  9. 2025FinanceReasoning: Benchmarking Financial Numerical Reasoning More Credible, Comprehensive and Challengingcited by 28
  10. 2022What do LLMs Know about Financial Markets? A Case Study on Reddit Market Sentiment Analysiscited by 27
  11. 2023Can Large Language Models Provide Security & Privacy Advice? Measuring the Ability of LLMs to Refute Misconceptionscited by 20
  12. 2024FinDVer: Explainable Claim Verification over Long and Hybrid-Content Financial Documentscited by 19
  13. 2023Integrating Stock Features and Global Information via Large Language Models for Enhanced Stock Return Predictioncited by 17
  14. 2024SEC-QA: A Systematic Evaluation Corpus for Financial QAcited by 17
  15. 2024Course-Correction: Safety Alignment Using Synthetic Preferencescited by 13
  16. 2025Expect the Unexpected: FailSafe Long Context QA for Financecited by 7
  17. 2025Open FinLLM Leaderboard: Towards Financial AI Readinesscited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 6.

8 papers.

  1. 2026FinVault: Benchmarking Financial Agent Safety in Execution-Grounded Environmentscited by 7
  2. 2025TradeTrap: Are LLM-based Trading Agents Truly Reliable and Faithful?cited by 2
  3. 2025Enhancing Trustworthiness with Mixed Precision: Benchmarks, Opportunities, and Challengescited by 0
  4. 2026Benchmarking Large Language Models for Quebec Insurance: From Closed-Book to Retrieval-Augmented Generationcited by 0
  5. 2026NextFund: A Unified Performance Tracking Platform for Agentic Portfolio Managementcited by 0
  6. 2026Confidence Estimation for Financial Vision-Language Models in Chart and Document Understandingcited by 0
  7. 2026Poisoning Agentic Alpha: Adversarial Vulnerabilities Across Roles and Architectures in Multi-Agent Trading Systemscited by 0
  8. 2026A Citation-Grounded Benchmark for Trustworthy Earnings Call Transcript Analysis with Large Language Modelscited by 0