From Accuracy to Auditability: A Survey of Determinism in Financial AI Systems
Not on the map. Why?
Abstract (arXiv)
Deploying machine learning in regulated financial environments -- credit risk, fraud detection, and anti-money laundering -- exposes critical vulnerabilities in algorithmic reproducibility. While early financial ML addressed statistical challenges such as backtest overfitting, deep neural networks and Generative AI have introduced mechanical nondeterminism rooted in hardware and architecture. This survey provides a systems perspective on reproducibility failures across three modalities now dominant in financial AI: tabular models (post-hoc explanation variance), graph networks (stochastic sampling and temporal asynchrony), and LLM-based agentic workflows (batch-dependent divergence and trajectory drift). We supplement the literature analysis with first-party experiments on public financial datasets -- quantifying explanation rank instability in credit scoring, prediction flip rates in GNN-based fraud detection, and tensor-parallel-induced output divergence in LLM entity extraction. We propose a layered evaluation framework linking modality-specific metrics (RBO, D_cos, TDI, PSD) to audit readiness, and report where these measures overlap rather than complement one another.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 27 of its 51 references that are in the map. First posted on arXiv 11 May 2026.
Builds on
27 of 51 references.
References · 27
- 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
- 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
- 2017Inductive Representation Learning on Large Graphscited by 6,284
- 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
- 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
- 2019Fast Graph Representation Learning with PyTorch Geometriccited by 2,414
- 2018Graph Convolutional Neural Networks for Web-Scale Recommender Systemscited by 1,133
- 2018Pitfalls of Graph Neural Network Evaluationcited by 994
- 2021Efficient Large-Scale Language Model Training on GPU Clusters Using Megatron-LMcited by 621
- 2020Temporal Graph Networks for Deep Learning on Dynamic Graphscited by 521
- 2019Fooling LIME and SHAP: Adversarial Attacks on Post hoc Explanation Methodscited by 440
15 more
- 2020Improving KernelSHAP: Practical Shapley Value Estimation via Linear Regressioncited by 153
- 2025Understanding and Mitigating Numerical Sources of Nondeterminism in LLM Inferencecited by 47
- 2025Nemotron-Cascade: Scaling Cascaded Reinforcement Learning for General-Purpose Reasoning Modelscited by 45
- 2021On the Reproducibility of Neural Network Predictionscited by 28
- 2025Assessing Consistency and Reproducibility in the Outputs of Large Language Models: Evidence Across Diverse Finance and Accounting Taskscited by 19
- 2026LLM-42: Enabling Determinism in LLM Inference with Verified Speculationcited by 14
- 2025Deterministic Inference across Tensor Parallel Sizes That Eliminates Training-Inference Mismatchcited by 13
- 2024Network Analytics for Anti-Money Laundering -- A Systematic Literature Review and Experimental Evaluationcited by 7
- 2026Replayable Financial Agents: A Determinism-Faithfulness Assurance Harness for Tool-Using LLM Agentscited by 7
- 2025LLM Output Drift: Cross-Provider Validation & Mitigation for Financial Workflowscited by 6
- 2025STED and Consistency Scoring: A Framework for Evaluating LLM Structured Output Reliabilitycited by 5
- 2025Evaluating the stability of model explanations in instance-dependent cost-sensitive credit scoringcited by 3
- 2024Towards Modeling Data Quality and Machine Learning Model Performancecited by 1
- 2026AEMA: Verifiable Evaluation Framework for Trustworthy and Controlled Agentic LLM Systemscited by 1
- 2026Normalisation and Initialisation Strategies for Graph Neural Networks in Blockchain Anomaly Detectioncited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Towards Explainable and Reliable AI in Finance
- 2025Detecting Financial Fraud with Hybrid Deep Learning: A Mix-of-Experts Approach to Sequential and Anomalous Patterns
- 2025Regulatory Graphs and GenAI for Real-Time Transaction Monitoring and Compliance Explanation in Banking
- 2025VERAFI: Verified Agentic Financial Intelligence through Neurosymbolic Policy Generation
- 2024FinRobot: An Open-Source AI Agent Platform for Financial Applications using Large Language Models
- 2026Toward Auditable Fraud Detection: Combining Graph Features, Model Explanations, and Agentic Case Investigation
- 2026CreditAudit: 2nd Dimension for LLM Evaluation and Selection
- 2025Understanding Structured Financial Data with LLMs: A Case Study on Fraud Detection