# Derandomized Novelty Detection with FDR Control via Conformal E-values

arXiv: https://arxiv.org/abs/2302.07294 · HTML: https://mlsanity.com/paper/2302.07294/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 50 references that are in the map.
- First posted on arXiv 14 Feb 2023.

## Builds on (most cited first, 17 of 17)

- [Conformalized Quantile Regression](https://mlsanity.com/paper/1905.03222/) (2019)
- [Classification with Valid and Adaptive Coverage](https://mlsanity.com/paper/2006.02544/) (2020)
- [Interpretable machine learning: definitions, methods, and applications](https://mlsanity.com/paper/1901.04592/) (2019)
- [Predictive inference with the jackknife+](https://mlsanity.com/paper/1905.02928/) (2019)
- [Testing for Outliers with Conformal p-values](https://mlsanity.com/paper/2104.08279/) (2021)
- [Controlling the false discovery rate via knockoffs](https://mlsanity.com/paper/1404.5609/) (2014)
- [Nested conformal prediction and quantile out-of-bag ensemble methods](https://mlsanity.com/paper/1910.10562/) (2019)
- [Veridical Data Science](https://mlsanity.com/paper/1901.08152/) (2019)
- [Predictive Inference Is Free with the Jackknife+-after-Bootstrap](https://mlsanity.com/paper/2002.09025/) (2020)
- [Prediction and outlier detection in classification problems](https://mlsanity.com/paper/1905.04396/) (2019)
- [Integrative conformal p-values for powerful out-of-distribution testing with labeled outliers](https://mlsanity.com/paper/2208.11111/) (2022)
- [Derandomized knockoffs: leveraging e-values for false discovery rate control](https://mlsanity.com/paper/2205.15461/) (2022)
- [Online Control of the False Coverage Rate and False Sign Rate](https://mlsanity.com/paper/1905.01059/) (2019)
- [Conditional calibration for false discovery rate control under dependence](https://mlsanity.com/paper/2007.10438/) (2020)
- [A Burden Shared is a Burden Halved: A Fairness-Adjusted Approach to Classification](https://mlsanity.com/paper/2110.05720/) (2021)
- [E-values as unnormalized weights in multiple testing](https://mlsanity.com/paper/2204.12447/) (2022)
- [Knockoffs with Side Information](https://mlsanity.com/paper/2001.07835/) (2020)

## Built on by (most cited first, 15 of 15)

- [Adaptive conformal classification with noisy labels](https://mlsanity.com/paper/2309.05092/) (2023)
- [Transductive conformal inference with adaptive scores](https://mlsanity.com/paper/2310.18108/) (2023)
- [WATCH: Adaptive Monitoring for AI Deployments via Weighted-Conformal Martingales](https://mlsanity.com/paper/2505.04608/) (2025)
- [Confidence on the Focal: Conformal Prediction with Selection-Conditional Coverage](https://mlsanity.com/paper/2403.03868/) (2024)
- [Handling Out-of-Distribution Data: A Survey](https://mlsanity.com/paper/2507.21160/) (2025)
- [Semi-Supervised Risk Control via Prediction-Powered Inference](https://mlsanity.com/paper/2412.11174/) (2024)
- [Conformalized Interval Arithmetic with Symmetric Calibration](https://mlsanity.com/paper/2408.10939/) (2024)
- [Model-agnostic Selective Labeling with Provable Statistical Guarantees](https://mlsanity.com/paper/2510.14581/) (2025)
- [A Conditional Distribution Equality Testing Framework using Deep Generative Learning](https://mlsanity.com/paper/2509.17729/) (2025)
- [Elements of Conformal Prediction](https://mlsanity.com/paper/2603.23923/) (2026)
- [Beyond Fixed False Discovery Rates: Post-Hoc Conformal Selection with E-Variables](https://mlsanity.com/paper/2604.11305/) (2026)
- [Data-light Uncertainty Set Merging with Admissibility](https://mlsanity.com/paper/2410.12201/) (2024)
- [On the Adversarial Robustness of Learning-based Conformal Novelty Detection](https://mlsanity.com/paper/2510.00463/) (2025)
- [Replicable Conformal Prediction](https://mlsanity.com/paper/2608.23638/) (2026)
- [When Identical Rows Disagree: From Benchmark Identifiability to Replication-Robust Anomaly Detection](https://mlsanity.com/paper/2609.29580/) (2026)

## Abstract (arXiv)

Conformal inference provides a general distribution-free method to rigorously calibrate the output of any machine learning algorithm for novelty detection. While this approach has many strengths, it has the limitation of being randomized, in the sense that it may lead to different results when analyzing twice the same data, and this can hinder the interpretation of any findings. We propose to make conformal inferences more stable by leveraging suitable conformal e-values instead of p-values to quantify statistical significance. This solution allows the evidence gathered from multiple analyses of the same data to be aggregated effectively while provably controlling the false discovery rate. Further, we show that the proposed method can reduce randomness without much loss of power compared to standard conformal inference, partly thanks to an innovative way of weighting conformal e-values based on additional side information carefully extracted from the same data. Simulations with synthetic and real data confirm this solution can be effective at eliminating random noise in the inferences obtained with state-of-the-art alternative techniques, sometimes also leading to higher power.