# A Distributional Framework for Behavioral Characterization and Risk Management of Large Language Models

arXiv: https://arxiv.org/abs/2610.12718 · HTML: https://mlsanity.com/paper/2610.12718/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 7 of its 18 references that are in the map.
- First posted on arXiv 8 Oct 2026.

## Builds on (most cited first, 7 of 7)

- [ProSA: Assessing and Understanding the Prompt Sensitivity of LLMs](https://mlsanity.com/paper/2410.12405/) (2024)
- [POSIX: A Prompt Sensitivity Index For Large Language Models](https://mlsanity.com/paper/2410.02185/) (2024)
- [Flaw or Artifact? Rethinking Prompt Sensitivity in Evaluating LLMs](https://mlsanity.com/paper/2509.01790/) (2025)
- [Statistical Hypothesis Testing for Auditing Robustness in Language Models](https://mlsanity.com/paper/2506.07947/) (2025)
- [Model Validation Practice in Banking: A Structured Approach for Predictive Models](https://mlsanity.com/paper/2410.13877/) (2024)
- [Understanding the Prompt Sensitivity](https://mlsanity.com/paper/2604.18389/) (2026)
- [Empirical Cumulative Distribution Function Clustering for LLM-based Agent System Analysis](https://mlsanity.com/paper/2602.16131/) (2026)

## Abstract (arXiv)

Large language models are stochastic by nature. Repeated executions under the same experimental conditions may produce different responses. We propose a framework that can be applied across the model lifecycle, from task design to residual behavior and continuous monitoring. The method consists of building an empirical response distribution estimated through Monte Carlo simulation. Semantic interventions are introduced to study how these distributions change. We illustrate the framework with two different tasks, borrower classification and music album recommendation, and quantify the resulting distributional changes. The framework is linked to model risk management and governance by measuring robustness and control effectiveness, and characterizing residual behavior. Since it relies only on observed responses, it can also be applied to black-box models accessed through commercial APIs.