# Anytime PAC-Bayes for Constrained Density-Ratio Networks under Covariate Shift

arXiv: https://arxiv.org/abs/2605.17212 · HTML: https://mlsanity.com/paper/2605.17212/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 2 of its 33 references that are in the map.
- First posted on arXiv 17 May 2026.

## Builds on (most cited first, 2 of 2)

- [Retiring Adult: New Datasets for Fair Machine Learning](https://mlsanity.com/paper/2108.04884/) (2021)
- [Density ratio model for multiple types of survival data with empirical likelihood](https://mlsanity.com/paper/2511.09398/) (2025)

## Abstract (arXiv)

A unified framework for learning under covariate shift is presented, in which a constrained density-ratio network approximates the Radon-Nikodym derivative r^star = dP/dQ and feeds an anytime PAC-Bayes generalization certificate. A change-of-measure identity decomposes the gap between target risk and importance-weighted source risk into a ratio-bias term governed by |rθ - r^star|L2(Q) and a generalization-gap term governed by the variability of the weighted loss. Normalization and moment-matching identities are enforced as hard integral constraints through an augmented-Lagrangian scheme, with a second-moment penalty controlling the effective sample size. PAC-Bayes is instantiated on the weighted risk in a fixed-time regime that yields Bernoulli-KL bounds, identifies the network-weighted Gibbs posterior as the unique KL-regularized minimizer, and quantifies stability under L2(Q) perturbations of the learned ratio, and is then strengthened by geometric peeling to an anytime certificate uniform in t ≥ tmin. A pre-registered two-campaign protocol combining a patch test against analytic ground truth with a real-data deployment validates the framework: the network produces calibrated ratios, reduces target 0/1 loss against unweighted ERM and classical direct ratio-estimation baselines, and attains the anytime certificate. A single fixed-time coverage failure is recorded, with per-split coverage aligning one-to-one with the magnitude of the label shift, confirming that the covariate-only assumption is operationally tight rather than a defect of the certificate.