mlsanityData to 7 Oct 2026

Amortized nonmyopic active search via deep imitation learning

arXiv 2405.15031 ↗First posted 23 May 2024cs.LGcited by 2

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

Active search formalizes a specialized active learning setting where the goal is to collect members of a rare, valuable class. The state-of-the-art algorithm approximates the optimal Bayesian policy in a budget-aware manner, and has been shown to achieve impressive empirical performance in previous work. However, even this approximate policy has a superlinear computational complexity with respect to the size of the search problem, rendering its application impractical in large spaces or in real-time systems where decisions must be made quickly. We study the amortization of this policy by training a neural network to learn to search. To circumvent the difficulty of learning from scratch, we appeal to imitation learning techniques to mimic the behavior of the expert, expensive-to-compute policy. Our policy network, trained on synthetic data, learns a beneficial search strategy that yields nonmyopic decisions carefully balancing exploration and exploitation. Extensive experiments demonstrate our policy achieves competitive performance at real-world tasks that closely approximates the expert's at a fraction of the cost, while outperforming cheaper baselines.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 25 of its 55 references that are in the map. First posted on arXiv 23 May 2024.

Builds on

25 of 55 references.

References · 25

  1. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  2. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,841
  3. 2017Billion-scale similarity search with GPUscited by 2,477
  4. 2016Learning to learn by gradient descent by gradient descentcited by 616
  5. 2014Semi-Supervised Learning with Deep Generative Modelscited by 494
  6. 2018GuacaMol: Benchmarking Models for De Novo Molecular Designcited by 294
  7. 2014Learning to Optimize via Information-Directed Samplingcited by 168
  8. 2015Deep Reinforcement Learning in Large Discrete Action Spacescited by 158
  9. 2021Deep Adaptive Design: Amortizing Sequential Bayesian Experimental Designcited by 103
  10. 2017Learning Active Learning from Datacited by 100
  11. 2017Learning how to Active Learn: A Deep Reinforcement Learning Approachcited by 79
  12. 2022Local Latent Space Bayesian Optimization over Structured Inputscited by 78
13 more
  1. 2020Graph Policy Network for Transferable Active Learning on Graphscited by 49
  2. 2022Optimizing Sequential Experimental Design with Deep Reinforcement Learningcited by 45
  3. 2015Upper-Confidence-Bound Algorithms for Active Learning in Multi-Armed Banditscited by 29
  4. 2020Similarity Search for Efficient Active Learning and Search of Rare Conceptscited by 24
  5. 2021Bayesian Sequential Optimal Experimental Design for Nonlinear Models Using Policy Gradient Reinforcement Learningcited by 22
  6. 2024Quality-Weighted Vendi Scores And Their Application To Diverse Experimental Designcited by 15
  7. 2023Amortized Inference for Gaussian Process Hyperparameters of Structured Kernelscited by 10
  8. 2022A Visual Active Search Framework for Geospatial Explorationcited by 8
  9. 2023A Partially Supervised Reinforcement Learning Framework for Visual Active Searchcited by 8
  10. 2022Nonmyopic Multiclass Active Search with Diminishing Returns for Diverse Discoverycited by 4
  11. 2022Adaptive Sampling for Discoverycited by 2
  12. 2020Guided Data Discovery in Interactive Visualizations via Active Searchcited by 1
  13. 2021Active Coveringcited by 1

Built on by

Cited by year

Citing papers per year: 2025: 2.

2 papers.

  1. 2025Online Feedback Efficient Active Target Discovery in Partially Observable Environmentscited by 1
  2. 2025Active Target Discovery under Uninformative Prior: The Power of Permanent and Transient Memorycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2020Active Imitation Learning with Noisy Guidance
  2. 2026Cost-Aware Diffusion Active Search
  3. 2019A New Framework for Query Efficient Active Imitation Learning
  4. 2025Sample-Efficient Expert Query Control in Active Imitation Learning via Conformal Prediction
  5. 2025Beyond-Expert Performance with Limited Demonstrations: Efficient Imitation Learning with Double Exploration
  6. 2019Bayesian policy selection using active inference
  7. 2023Active Policy Improvement from Multiple Black-box Oracles
  8. 2019Few-Shot Bayesian Imitation Learning with Logical Program Policies