# Sequential Confidence Sets for Coverage-Constrained Conformal Model Selection

arXiv: https://arxiv.org/abs/2609.28522 · HTML: https://mlsanity.com/paper/2609.28522/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 7 of its 37 references that are in the map.
- First posted on arXiv 22 Sep 2026.

## Builds on (most cited first, 7 of 7)

- [Conformal prediction after data-dependent model selection](https://mlsanity.com/paper/2408.07066/) (2024)
- [Winners with Confidence: Discrete Argmin Inference with an Application to Model Selection](https://mlsanity.com/paper/2408.02060/) (2024)
- [Anytime-Valid Conformal Risk Control](https://mlsanity.com/paper/2602.04364/) (2026)
- [Locally minimax optimal confidence sets for the best model](https://mlsanity.com/paper/2503.21639/) (2025)
- [Localized conformal model selection](https://mlsanity.com/paper/2602.19284/) (2026)
- [Conformal Prediction Assessment: A Framework for Conditional Coverage Evaluation and Selection](https://mlsanity.com/paper/2603.27189/) (2026)
- [Online Conformal Model Selection for Nonstationary Time Series](https://mlsanity.com/paper/2506.05544/) (2025)

## Abstract (arXiv)

Modern conformal forecasting systems often maintain several adaptive pipelines that differ in base forecasters, conformity scores, calibration windows, and update rules. Comparing them is difficult because coverage is a hard constraint, whereas efficiency should be optimized only among feasible pipelines. We formulate this problem as sequential inference for a stochastic constrained argmin. At each time, the target is the set of minimum-cost pipelines satisfying multiple prefix-average conditional miscoverage constraints. We introduce Coverage-Constrained Sequential Model Confidence Sets (CC-SMCS), which separate certifiably feasible, possibly feasible, and possibly constrained-optimal pipelines. Using simultaneous martingale confidence sequences, CC-SMCS projects a rectangular confidence region onto the constrained argmin and admits an exact closed-form rule. With probability at least 1-δ, it contains every constrained-optimal pipeline simultaneously over all times. This finite-sample guarantee requires no stationarity or mixing assumptions and remains valid under data-dependent stopping. We also establish an impossibility result for safe certification at the coverage boundary and extend the construction to delayed multi-horizon feedback and outcome-dependent efficiency objectives.