# Taming Variability: Randomized and Bootstrapped Conformal Risk Control for LLMs

arXiv: https://arxiv.org/abs/2509.23007 · HTML: https://mlsanity.com/paper/2509.23007/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Sep 2025.

## Built on by (most cited first, 2 of 2)

- [Beyond Aggregate Risk: Role-Stratified Conformal Risk Control for LLM Tool Calls](https://mlsanity.com/paper/2607.24343/) (2026)
- [SIGMA: Scalable Spectral Insights for LLM Model Collapse](https://mlsanity.com/paper/2601.03385/) (2026)

## Abstract (arXiv)

We transform the randomness of LLMs into precise assurances using an actuator at the API interface that applies a user-defined risk constraint in finite samples via Conformal Risk Control (CRC). This label-free and model-agnostic actuator manages ship/abstain/escalate actions based solely on a scalar score from opaque outputs. We enhance CRC's computational efficiency and robustness through Batched Bootstrap CRC (BB-CRC) and Randomized Batched Weighted-Average CRC (RBWA-CRC), reducing calibration calls and stabilizing thresholds while maintaining statistical validity. Additionally, we present a semantic quantification method grounded in gram matrix geometry, resulting in interpretable signal and metric design. Together these pieces deliver principled randomness control for LLM hallucination mitigation and LLM-as-judge reliability. Our framework is assessed using four datasets, demonstrating its efficacy in enhancing factual accuracy and measuring LLM-as-judge performance, yielding a simplified and computationally efficient control layer that converts variability into statistical validity.