# Bayesian Active Learning in the Presence of Nuisance Parameters

arXiv: https://arxiv.org/abs/2310.14968 · HTML: https://mlsanity.com/paper/2310.14968/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 47 references that are in the map.
- First posted on arXiv 23 Oct 2023.

## Builds on (most cited first, 29 of 29)

- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Acceleration](https://mlsanity.com/paper/1809.11165/) (2018)
- [Characterizing and Avoiding Negative Transfer](https://mlsanity.com/paper/1811.09751/) (2018)
- [Max-value Entropy Search for Efficient Bayesian Optimization](https://mlsanity.com/paper/1703.01968/) (2017)
- [Predictive Entropy Search for Efficient Global Optimization of Black-box Functions](https://mlsanity.com/paper/1406.2541/) (2014)
- [Bayesian Model-Agnostic Meta-Learning](https://mlsanity.com/paper/1806.03836/) (2018)
- [Recasting Gradient-Based Meta-Learning as Hierarchical Bayes](https://mlsanity.com/paper/1801.08930/) (2018)
- [Modern Bayesian Experimental Design](https://mlsanity.com/paper/2302.14545/) (2023)
- [Variational Bayesian Optimal Experimental Design](https://mlsanity.com/paper/1903.05480/) (2019)
- [Meta-Learning Probabilistic Inference For Prediction](https://mlsanity.com/paper/1805.09921/) (2018)
- [On Nesting Monte Carlo Estimators](https://mlsanity.com/paper/1709.06181/) (2017)
- [On Statistical Bias In Active Learning: How and When To Fix It](https://mlsanity.com/paper/2101.11665/) (2021)
- [Adversarially Robust Optimization with Gaussian Processes](https://mlsanity.com/paper/1810.10775/) (2018)
- [Implicit Deep Adaptive Design: Policy-Based Experimental Design without Likelihoods](https://mlsanity.com/paper/2111.02329/) (2021)
- [A Unified Stochastic Gradient Approach to Designing Bayesian-Optimal Experiments](https://mlsanity.com/paper/1911.00294/) (2019)
- [Misspecified Gaussian Process Bandit Optimization](https://mlsanity.com/paper/2111.05008/) (2021)
- [Optimizing Sequential Experimental Design with Deep Reinforcement Learning](https://mlsanity.com/paper/2202.00821/) (2022)
- [Bayesian decision-making under misspecified priors with applications to meta-learning](https://mlsanity.com/paper/2107.01509/) (2021)
- [GLASSES: Relieving The Myopia Of Bayesian Optimisation](https://mlsanity.com/paper/1510.06299/) (2015)
- [Self-Correcting Bayesian Optimization through Bayesian Active Learning](https://mlsanity.com/paper/2304.11005/) (2023)
- [Policy-Based Bayesian Experimental Design for Non-Differentiable Implicit Models](https://mlsanity.com/paper/2203.04272/) (2022)
- [Causal Entropy Optimization](https://mlsanity.com/paper/2208.10981/) (2022)
- [Robust Expected Information Gain for Optimal Bayesian Experimental Design Using Ambiguity Sets](https://mlsanity.com/paper/2205.09914/) (2022)
- [Bayesian decision-theoretic design of experiments under an alternative model](https://mlsanity.com/paper/1909.12570/) (2019)
- [Characterizing the robustness of Bayesian adaptive experimental designs to active learning bias](https://mlsanity.com/paper/2205.13698/) (2022)
- [Metrics for Bayesian Optimal Experiment Design under Model Misspecification](https://mlsanity.com/paper/2304.07949/) (2023)
- [Corruption Robust Active Learning](https://mlsanity.com/paper/2106.11220/) (2021)
- [Robustness of Bayesian Pool-based Active Learning Against Prior Misspecification](https://mlsanity.com/paper/1603.09050/) (2016)
- [Knowing what to know: Implications of the choice of prior distribution on the behavior of adaptive design optimization](https://mlsanity.com/paper/2303.12683/) (2023)

## Built on by (most cited first, 6 of 6)

- [Generalization Bounds and Stopping Rules for Learning with Self-Selected Data](https://mlsanity.com/paper/2505.07367/) (2025)
- [Proxy-informed Bayesian transfer learning with unknown sources](https://mlsanity.com/paper/2411.03263/) (2024)
- [Representative, Informative, and De-Amplifying: Requirements for Robust Bayesian Active Learning under Model Misspecification](https://mlsanity.com/paper/2506.07805/) (2025)
- [Diagnose Before You Compress: Prediction-Independent Bottleneck Witness Refinement for LLM Serving Traces](https://mlsanity.com/paper/2608.00423/) (2026)
- [Learning What Matters: Adaptive Information-Theoretic Objectives for Robot Exploration](https://mlsanity.com/paper/2605.12084/) (2026)
- [Revealing After Overwriting: An Exponential POMDP OPE Lower Bound under History-Dependent Logging](https://mlsanity.com/paper/2610.05063/) (2026)

## Abstract (arXiv)

In many settings, such as scientific inference, optimization, and transfer learning, the learner has a well-defined objective, which can be treated as estimation of a target parameter, and no intrinsic interest in characterizing the entire data-generating process. Usually, the learner must also contend with additional sources of uncertainty or variables -- with nuisance parameters. Bayesian active learning, or sequential optimal experimental design, can straightforwardly accommodate the presence of nuisance parameters, and so is a natural active learning framework for such problems. However, the introduction of nuisance parameters can lead to bias in the Bayesian learner's estimate of the target parameters, a phenomenon we refer to as negative interference. We characterize the threat of negative interference and how it fundamentally changes the nature of the Bayesian active learner's task. We show that the extent of negative interference can be extremely large, and that accurate estimation of the nuisance parameters is critical to reducing it. The Bayesian active learner is confronted with a dilemma: whether to spend a finite acquisition budget in pursuit of estimation of the target or of the nuisance parameters. Our setting encompasses Bayesian transfer learning as a special case, and our results shed light on the phenomenon of negative transfer between learning environments.