# Risk-based test framework for LLM features in regulated software

arXiv: https://arxiv.org/abs/2601.17292 · HTML: https://mlsanity.com/paper/2601.17292/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Jan 2026.

## Built on by (most cited first, 1 of 1)

- [Same Question, Different Answer? Measuring and Mitigating Prompt Privilege for Equitable AI Access](https://mlsanity.com/paper/2608.08942/) (2026)

## Abstract (arXiv)

Large language models are increasingly embedded in regulated and safety-critical software, including clinical research platforms and healthcare information systems. While these features enable natural language search, summarization, and configuration assistance, they introduce risks such as hallucinations, harmful or out-of-scope advice, privacy and security issues, bias, instability under change, and adversarial misuse. Prior work on machine learning testing and AI assurance offers useful concepts but limited guidance for interactive, product-embedded assistants. This paper proposes a risk-based testing framework for LLM features in regulated software: a six-category risk taxonomy, a layered test strategy mapping risks to concrete tests across guardrail, orchestration, and system layers, and a case study applying the approach to a Knowledgebase assistant in a clinical research platform.