# Estimate Level Adjustment For Inference With Proxies Under Random Distribution Shifts

arXiv: https://arxiv.org/abs/2605.06484 · HTML: https://mlsanity.com/paper/2605.06484/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 50 references that are in the map.
- First posted on arXiv 7 May 2026.

## Builds on (most cited first, 13 of 13)

- [WILDS: A Benchmark of in-the-Wild Distribution Shifts](https://mlsanity.com/paper/2012.07421/) (2020)
- [Nuanced Metrics for Measuring Unintended Bias with Real Data for Text Classification](https://mlsanity.com/paper/1903.04561/) (2019)
- [Detecting and Correcting for Label Shift with Black Box Predictors](https://mlsanity.com/paper/1802.03916/) (2018)
- [Multiaccuracy: Black-Box Post-Processing for Fairness in Classification](https://mlsanity.com/paper/1805.12317/) (2018)
- [Choosing a Proxy Metric from Past Experiments](https://mlsanity.com/paper/2309.07893/) (2023)
- [Inferring the Long-Term Causal Effects of Long-Term Treatments from Short-Term Experiments](https://mlsanity.com/paper/2311.08527/) (2023)
- [Calibrate-Extrapolate: Rethinking Prevalence Estimation with Black Box Classifiers](https://mlsanity.com/paper/2401.09329/) (2024)
- [Multiply Robust Estimation for Local Distribution Shifts with Multiple Domains](https://mlsanity.com/paper/2402.14145/) (2024)
- [Learning the Covariance of Treatment Effects Across Many Weak Experiments](https://mlsanity.com/paper/2402.17637/) (2024)
- [Confidence Intervals for Nonparametric Empirical Bayes Analysis](https://mlsanity.com/paper/1902.02774/) (2019)
- [CTRL Your Shift: Clustered Transfer Residual Learning for Many Small Datasets](https://mlsanity.com/paper/2508.11144/) (2025)
- [Data Harmonization Via Regularized Nonparametric Mixing Distribution Estimation](https://mlsanity.com/paper/2110.06077/) (2021)
- [Transfer Learning Through Conditional Quantile Matching](https://mlsanity.com/paper/2602.02358/) (2026)

## Built on by (most cited first, 1 of 1)

- [proxymate: Diagnosis and Adjustment of Proxy Estimates for Reliable Inference](https://mlsanity.com/paper/2607.24401/) (2026)

## Abstract (arXiv)

In many scientific domains, including experimentation, researchers rely on measurements of proxy outcomes to achieve faster and more frequent reads, especially when the primary outcome of interest is challenging to measure directly. While proxies offer a more readily accessible observation for inference, the ultimate goal is to draw statistical inferences about the primary outcome parameter and proxy data are typically imperfect in some ways. To correct for these imperfections, current statistical inference methods often depend on strict identifying assumptions (such as surrogacy, covariate/label shift, or missingness assumptions). These assumptions can be difficult to validate and may be violated by various additional sources of distribution shift, potentially leading to biased parameter estimates and miscalibrated uncertainty quantification. We introduce an estimate-level framework, inspired by domain adaptation techniques, to empirically calibrate proxy-based inference. This framework models the proxy-primary metric discrepancy as a random effect at the parameter level, estimating its distribution from aggregated historical observations across past domains (e.g., experiments, time periods, or distinct segments). This method avoids the requirement for retaining individual-level response data. Additionally, this adjustment can be layered on top of existing proxy-correction methods (such as prediction-powered inference or importance weighting) to account for additional biases not addressed by those corrections. To manage uncertainty when the number of historical domains is limited, we provide both a method-of-moments estimator and a domain bootstrap procedure. We further validate this approach using publicly available datasets and real-world experiments.