mlsanityData to 13 Oct 2026

Meta-learning Adaptive Deep Kernel Gaussian Processes for Molecular Property Prediction

arXiv 2205.02708 ↗First posted 5 May 2022cs.LGcited by 21

Not on the map. Why?

Abstract (arXiv)

We propose Adaptive Deep Kernel Fitting with Implicit Function Theorem (ADKF-IFT), a novel framework for learning deep kernel Gaussian processes (GPs) by interpolating between meta-learning and conventional deep kernel learning. Our approach employs a bilevel optimization objective where we meta-learn generally useful feature representations across tasks, in the sense that task-specific GP models estimated on top of such features achieve the lowest possible predictive loss on average. We solve the resulting nested optimization problem using the implicit function theorem (IFT). We show that our ADKF-IFT framework contains previously proposed Deep Kernel Learning (DKL) and Deep Kernel Transfer (DKT) as special cases. Although ADKF-IFT is a completely general method, we argue that it is especially well-suited for drug discovery problems and demonstrate that it significantly outperforms previous state-of-the-art methods on a variety of real-world few-shot molecular property prediction tasks and out-of-domain molecular property prediction and optimization tasks.

Key facts. Cited by 21 papers in the map (data through 13 Oct 2026). Builds on 48 of its 84 references that are in the map. First posted on arXiv 5 May 2022.

Builds on

48 of 84 references.

References · 48

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,093
  2. 2013Intriguing properties of neural networkscited by 5,137
  3. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,870
  4. 2018How Powerful are Graph Neural Networks?cited by 4,373
  5. 2017Neural Message Passing for Quantum Chemistrycited by 3,471
  6. 2017Prototypical Networks for Few-shot Learningcited by 3,011
  7. 2016Matching Networks for One Shot Learningcited by 2,251
  8. 2017MoleculeNet: A Benchmark for Molecular Machine Learningcited by 1,043
  9. 2019Strategies for Pre-training Graph Neural Networkscited by 965
  10. 2018A Tutorial on Bayesian Optimizationcited by 739
  11. 2019Meta-Learning with Implicit Gradientscited by 523
  12. 2020Principal Neighbourhood Aggregation for Graph Netscited by 478
36 more
  1. 2019Meta-Learning with Differentiable Convex Optimizationcited by 461
  2. 2015Deep Kernel Learningcited by 441
  3. 2020Rethinking Few-Shot Image Classification: a Good Embedding Is All You Need?cited by 436
  4. 2018Conditional Neural Processescited by 423
  5. 2020Simple and Principled Uncertainty Estimation with Deterministic Deep Learning via Distance Awarenesscited by 389
  6. 2019Meta-Dataset: A Dataset of Datasets for Learning to Learn from Few Examplescited by 314
  7. 2019Optimizing Millions of Hyperparameters by Implicit Differentiationcited by 303
  8. 2016Hyperparameter optimization with approximate gradientcited by 232
  9. 2018Learning to Propagate Labels: Transductive Propagation Network for Few-shot Learningcited by 209
  10. 2019Edge-labeling Graph Neural Network for Few-shot Learningcited by 114
  11. 2016Stochastic Variational Deep Kernel Learningcited by 93
  12. 2020Molecule Attention Transformercited by 92
  13. 2016Low Data Drug Discovery with One-shot Learningcited by 91
  14. 2021On Feature Collapse and Deep Kernel Learning for Single Forward Pass Uncertaintycited by 87
  15. 2016Multi-Information Source Optimizationcited by 83
  16. 2021Few-Shot Graph Learning for Molecular Property Predictioncited by 81
  17. 2021The Promises and Pitfalls of Deep Kernel Learningcited by 81
  18. 2014Manifold Gaussian Processes for Regressioncited by 79
  19. 2020Leveraging the Feature Distribution in Transfer-based Few-Shot Learningcited by 73
  20. 2015Scalable Gradient-Based Tuning of Continuous Regularization Hyperparameterscited by 71
  21. 2019Meta-Curvaturecited by 70
  22. 2021Few-Shot Bayesian Optimization with Deep Kernel Surrogatescited by 70
  23. 2021iDARTS: Differentiable Architecture Search with Stochastic Implicit Gradientscited by 51
  24. 2017Adversarial Examples, Uncertainty, and Transfer Testing Robustness in Gaussian Process Hybrid Deep Networkscited by 48
  25. 2019Self-Supervised Learning For Few-Shot Image Classificationcited by 44
  26. 2021Property-Aware Relation Networks for Few-Shot Molecular Property Predictioncited by 41
  27. 2019On Solving Minimax Optimization Locally: A Follow-the-Ridge Approachcited by 34
  28. 2021DOCKSTRING: easy molecular docking yields better benchmarks for ligand designcited by 33
  29. 2020Bayesian Few-Shot Classification with One-vs-Each P'olya-Gamma Augmented Gaussian Processescited by 32
  30. 2019Modular Meta-Learning with Shrinkagecited by 31
  31. 2021Transfer Learning with Gaussian Processes for Bayesian Optimizationcited by 26
  32. 2021Meta Two-Sample Testing: Learning Kernels for Testing with Limited Datacited by 18
  33. 2018Efficient Gaussian Process Classification Using Polya-Gamma Data Augmentationcited by 17
  34. 2019Adaptive Deep Kernel Learningcited by 15
  35. 2021Scalable One-Pass Optimisation of High-Dimensional Weight-Update Hyperparameters by Implicit Differentiationcited by 7
  36. 2021Gaussian Process Meta Few-shot Classifier Learning via Linear Discriminant Laplace Approximationcited by 3

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2022: 1; 2023: 9; 2024: 7; 2025: 3; 2026: 1.

21 papers.

Sort:

  1. 2023Enhancing Activity Prediction Models in Drug Discovery with the Ability to Understand Human Languagecited by 44
  2. 2023End-to-End Meta-Bayesian Optimisation with Transformer Neural Processescited by 31
  3. 2022Gradient-based Bi-level Optimization for Deep Learning: A Surveycited by 26
  4. 2023Context-enriched molecule representations improve few-shot drug discoverycited by 24
  5. 2023A Survey of Imbalanced Learning on Graphs: Problems, Techniques, and Future Directionscited by 15
  6. 2025Large language models as uncertainty-calibrated optimizers for experimental discoverycited by 12
  7. 2023In-Context Learning for Few-Shot Molecular Property Predictioncited by 7
  8. 2024Impact of Domain Knowledge and Multi-Modality on Intelligent Molecular Property Prediction: A Systematic Surveycited by 6
  9. 2023PACIA: Parameter-Efficient Adapter for Few-Shot Molecular Property Predictioncited by 4
  10. 2024Pin-Tuning: Parameter-Efficient In-Context Tuning for Few-Shot Molecular Property Predictioncited by 4
  11. 2024TwinBooster: Synergising Large Language Models with Barlow Twins and Gradient Boosting for Enhanced Molecular Property Predictioncited by 3
  12. 2024Learning to Learn with Contrastive Meta-Objectivecited by 3
9 more
  1. 2024Knowledge-enhanced Relation Graph and Task Sampling for Few-shot Molecular Property Predictioncited by 2
  2. 2023Leveraging Task Structures for Improved Identifiability in Neural Network Representationscited by 1
  3. 2023It HAS to be Subjective: Human Annotator Simulation via Zero-shot Density Estimationcited by 1
  4. 2025Rethinking the generalization of drug target affinity prediction algorithms via similarity aware evaluationcited by 1
  5. 2023Target-Free Compound Activity Prediction via Few-Shot Learningcited by 0
  6. 2024A Strong Baseline for Molecular Few-Shot Learningcited by 0
  7. 2024Contextual Representation Anchor Network to Alleviate Selection Bias in Few-Shot Drug Discoverycited by 0
  8. 2025Meta-Learning Theory-Informed Inductive Biases using Deep Kernel Gaussian Processescited by 0
  9. 2026CalibHyper: Chance-Corrected Relational Hypergraphs for Few-Shot Molecular Property Predictioncited by 0