# Batch Multi-Fidelity Active Learning with Budget Constraints

arXiv: https://arxiv.org/abs/2210.12704 · HTML: https://mlsanity.com/paper/2210.12704/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 45 references that are in the map.
- First posted on arXiv 23 Oct 2022.

## Builds on (most cited first, 13 of 13)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Deep Bayesian Active Learning with Image Data](https://mlsanity.com/paper/1703.02910/) (2017)
- [Deep Batch Active Learning by Diverse, Uncertain Gradient Lower Bounds](https://mlsanity.com/paper/1906.03671/) (2019)
- [BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learning](https://mlsanity.com/paper/1906.08158/) (2019)
- [Adversarial Active Learning for Deep Networks: a Margin Based Approach](https://mlsanity.com/paper/1802.09841/) (2018)
- [Discriminative Active Learning](https://mlsanity.com/paper/1907.06347/) (2019)
- [Deep Active Learning over the Long Tail](https://mlsanity.com/paper/1711.00941/) (2017)
- [Bayesian Experimental Design for Implicit Models by Mutual Information Neural Estimation](https://mlsanity.com/paper/2002.08129/) (2020)
- [Deep Multi-Fidelity Active Learning of High-dimensional Outputs](https://mlsanity.com/paper/2012.00901/) (2020)
- [TrimTuner: Efficient Optimization of Machine Learning Jobs in the Cloud via Sub-Sampling](https://mlsanity.com/paper/2011.04726/) (2020)
- [Batch Multi-Fidelity Bayesian Optimization with Deep Auto-Regressive Networks](https://mlsanity.com/paper/2106.09884/) (2021)

## Built on by (most cited first, 20 of 20)

- [A Survey on Deep Active Learning: Recent Advances and New Frontiers](https://mlsanity.com/paper/2405.00334/) (2024)
- [Local Search GFlowNets](https://mlsanity.com/paper/2310.02710/) (2023)
- [Multi-Resolution Active Learning of Fourier Neural Operators](https://mlsanity.com/paper/2309.16971/) (2023)
- [Multi-Fidelity Active Learning with GFlowNets](https://mlsanity.com/paper/2306.11715/) (2023)
- [Multi-Fidelity Residual Neural Processes for Scalable Surrogate Modeling](https://mlsanity.com/paper/2402.18846/) (2024)
- [Disentangled Multi-Fidelity Deep Bayesian Active Learning](https://mlsanity.com/paper/2305.04392/) (2023)
- [Looking Backward: Retrospective Backward Synthesis for Goal-Conditioned GFlowNets](https://mlsanity.com/paper/2406.01150/) (2024)
- [Adaptive Batch Sizes for Active Learning A Probabilistic Numerics Approach](https://mlsanity.com/paper/2306.05843/) (2023)
- [Interactive Multi-fidelity Learning for Cost-effective Adaptation of Language Model with Sparse Human Supervision](https://mlsanity.com/paper/2310.20153/) (2023)
- [Active Learning with Selective Time-Step Acquisition for PDEs](https://mlsanity.com/paper/2511.18107/) (2025)
- [Diffusion-Generative Multi-Fidelity Learning for Physical Simulation](https://mlsanity.com/paper/2311.05606/) (2023)
- [A Quadrature Approach for General-Purpose Batch Bayesian Optimization via Probabilistic Lifting](https://mlsanity.com/paper/2404.12219/) (2024)
- [Aligned Multi Objective Optimization](https://mlsanity.com/paper/2502.14096/) (2025)
- [Enhancing RAG with Active Learning on Conversation Records: Reject Incapables and Answer Capables](https://mlsanity.com/paper/2502.09073/) (2025)
- [Active Domain Knowledge Acquisition with 100-Dollar Budget: Enhancing LLMs via Cost-Efficient, Expert-Involved Interaction in Sensitive Domains](https://mlsanity.com/paper/2508.17202/) (2025)
- [Future Aware Safe Active Learning of Time Varying Systems using Gaussian Processes](https://mlsanity.com/paper/2405.10581/) (2024)
- [ConBatch-BAL: Batch Bayesian Active Learning under Budget Constraints](https://mlsanity.com/paper/2507.04929/) (2025)
- [Multi-fidelity Batch Active Learning for Gaussian Process Classifiers](https://mlsanity.com/paper/2510.08865/) (2025)
- [Goal-Oriented Influence-Maximizing Data Acquisition for Learning and Optimization](https://mlsanity.com/paper/2602.19578/) (2026)
- [One Human, N Agents: Audit-Budget Allocation for LLM Agent Fleets under Miscalibrated, Correlated Confidence](https://mlsanity.com/paper/2607.28317/) (2026)

## Abstract (arXiv)

Learning functions with high-dimensional outputs is critical in many applications, such as physical simulation and engineering design. However, collecting training examples for these applications is often costly, e.g. by running numerical solvers. The recent work (Li et al., 2022) proposes the first multi-fidelity active learning approach for high-dimensional outputs, which can acquire examples at different fidelities to reduce the cost while improving the learning performance. However, this method only queries at one pair of fidelity and input at a time, and hence has a risk to bring in strongly correlated examples to reduce the learning efficiency. In this paper, we propose Batch Multi-Fidelity Active Learning with Budget Constraints (BMFAL-BC), which can promote the diversity of training examples to improve the benefit-cost ratio, while respecting a given budget constraint for batch queries. Hence, our method can be more practically useful. Specifically, we propose a novel batch acquisition function that measures the mutual information between a batch of multi-fidelity queries and the target function, so as to penalize highly correlated queries and encourages diversity. The optimization of the batch acquisition function is challenging in that it involves a combinatorial search over many fidelities while subject to the budget constraint. To address this challenge, we develop a weighted greedy algorithm that can sequentially identify each (fidelity, input) pair, while achieving a near (1 - 1/e)-approximation of the optimum. We show the advantage of our method in several computational physics and engineering applications.