# The Price of Self-Calibration: Exact Evidence Budgets and Manufactured Blind Sets in Adaptive Monitoring

arXiv: https://arxiv.org/abs/2609.23254 · HTML: https://mlsanity.com/paper/2609.23254/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 7 of its 26 references that are in the map.
- First posted on arXiv 19 Sep 2026.

## Builds on (most cited first, 7 of 7)

- [Adaptive Conformal Inference Under Distribution Shift](https://mlsanity.com/paper/2106.00170/) (2021)
- [Game-theoretic statistics and safe anytime-valid inference](https://mlsanity.com/paper/2210.01948/) (2022)
- [Conformal PID Control for Time Series Prediction](https://mlsanity.com/paper/2307.16895/) (2023)
- [Testing For Distribution Shifts with Conditional Conformal Test Martingales](https://mlsanity.com/paper/2602.13848/) (2026)
- [Between Resolution Collapse and Variance Inflation: Weighted Conformal Anomaly Detection in Low-Data Regimes](https://mlsanity.com/paper/2603.23205/) (2026)
- [The Boiling Frog Threshold: Criticality and Blindness in World Model-Based Anomaly Detection Under Gradual Drift](https://mlsanity.com/paper/2603.08455/) (2026)
- [Data-Driven Probabilistic Fault Detection and Identification via Density Flow Matching](https://mlsanity.com/paper/2603.25982/) (2026)

## Abstract (arXiv)

Self-calibrating monitors adapt their threshold online to guarantee a prescribed long-run false-alarm rate under arbitrary drift. We compute the price of that guarantee, stating every law with its exact domain of validity. First, the guarantee is an accounting identity, insensitive to what the monitor is meant to detect. Two evidence identities make the cost exact for the online quantile tracker: a persistent step of height δ yields excess alarm mass within one alarm of δ/η, and exactly δ/η pathwise when δ is a lattice multiple of the gain η; a ramp of slope c yields a stationary excess rate of exactly c/η, independent of accumulated size, up to a boundary c=η(1-α) coinciding with the alarm-rate cap. Second, the certificate's own fluctuation obeys an exact law: the windowed alarm rate has standard deviation of order 1/L, not the binomial 1/√L, since the windowed mass telescopes to a difference of a tight internal state; the closed-form constant is validated with no fitted parameter. Detectors calibrated on the binomial scale are miscalibrated by √ηvarphi(q0)L, and correct calibration turns detection windows from quadratic to linear in the inverse fault speed. Third, any monitor required to tolerate a drift class D is blind, at any horizon and for any rule, to every fault in D-D; the proof is a deliberately elementary two-point argument and the contribution is the object it identifies: for speed-bounded classes the blind set is exactly the doubled-speed class, and the tracker absorbs a speed class fixed by its own gain, so that under a certification regime declaring absorbed drift normal, the monitor manufactures D. An exact Gaussian projection bound, sharper than Pinsker and never vacuous, quantifies power outside it.