# LLM Output Drift: Cross-Provider Validation & Mitigation for Financial Workflows

arXiv: https://arxiv.org/abs/2511.07585 · HTML: https://mlsanity.com/paper/2511.07585/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 48 references that are in the map.
- First posted on arXiv 10 Nov 2025.

## Builds on (most cited first, 5 of 5)

- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [Non-Determinism of "Deterministic" LLM Settings](https://mlsanity.com/paper/2408.04667/) (2024)
- [DocFinQA: A Long-Context Financial Reasoning Dataset](https://mlsanity.com/paper/2401.06915/) (2024)
- [SEC-QA: A Systematic Evaluation Corpus for Financial QA](https://mlsanity.com/paper/2406.14394/) (2024)
- [Reproducibility, Replicability, and Repeatability: A survey of reproducible research with a focus on high performance computing](https://mlsanity.com/paper/2402.07530/) (2024)

## Built on by (most cited first, 7 of 7)

- [Replayable Financial Agents: A Determinism-Faithfulness Assurance Harness for Tool-Using LLM Agents](https://mlsanity.com/paper/2601.15322/) (2026)
- [Uncertainty Propagation in LLM-Based Systems](https://mlsanity.com/paper/2604.23505/) (2026)
- [The Dice Roll Method: A Standardized Protocol for Repeated-Query Auditing of Large Language Model Brand Recommendations](https://mlsanity.com/paper/2609.04047/) (2026)
- [LLM Readiness Harness: Evaluation, Observability, and CI Gates for LLM/RAG Applications](https://mlsanity.com/paper/2603.27355/) (2026)
- [From Accuracy to Auditability: A Survey of Determinism in Financial AI Systems](https://mlsanity.com/paper/2605.23955/) (2026)
- [Grading the Graders: Verification Autonomy Levels (L0-L5) for LLM Reasoning](https://mlsanity.com/paper/2608.19009/) (2026)
- [SiliconBench: Speed, Memory, and Fidelity for LLM Serving on Unified-Memory Desktops](https://mlsanity.com/paper/2609.19169/) (2026)

## Abstract (arXiv)

Financial institutions deploy Large Language Models (LLMs) for reconciliations, regulatory reporting, and client communications, but nondeterministic outputs (output drift) undermine auditability and trust. We quantify drift across five model architectures (7B-120B parameters) on regulated financial tasks, revealing a stark inverse relationship: smaller models (Granite-3-8B, Qwen2.5-7B) achieve 100% output consistency at T=0.0, while GPT-OSS-120B exhibits only 12.5% consistency (95% CI: 3.5-36.0%) regardless of configuration (p<0.0001, Fisher's exact test). This finding challenges conventional assumptions that larger models are universally superior for production deployment. Our contributions include: (i) a finance-calibrated deterministic test harness combining greedy decoding (T=0.0), fixed seeds, and SEC 10-K structure-aware retrieval ordering; (ii) task-specific invariant checking for RAG, JSON, and SQL outputs using finance-calibrated materiality thresholds (plus or minus 5%) and SEC citation validation; (iii) a three-tier model classification system enabling risk-appropriate deployment decisions; and (iv) an audit-ready attestation system with dual-provider validation. We evaluated five models (Qwen2.5-7B via Ollama, Granite-3-8B via IBM watsonx.ai, Llama-3.3-70B, Mistral-Medium-2505, and GPT-OSS-120B) across three regulated financial tasks. Across 480 runs (n=16 per condition), structured tasks (SQL) remain stable even at T=0.2, while RAG tasks show drift (25-75%), revealing task-dependent sensitivity. Cross-provider validation confirms deterministic behavior transfers between local and cloud deployments. We map our framework to Financial Stability Board (FSB), Bank for International Settlements (BIS), and Commodity Futures Trading Commission (CFTC) requirements, demonstrating practical pathways for compliance-ready AI deployments.