mlsanityData to 7 Oct 2026

Calibration and generalizability of probabilistic models on low-data chemical datasets with DIONYSUS

arXiv 2212.01574 ↗First posted 3 Dec 2022cs.CE cs.AIcited by 10

Not on the map. Why?

Abstract (arXiv)

Deep learning models that leverage large datasets are often the state of the art for modelling molecular properties. When the datasets are smaller (< 2000 molecules), it is not clear that deep learning approaches are the right modelling tool. In this work we perform an extensive study of the calibration and generalizability of probabilistic machine learning models on small chemical datasets. Using different molecular representations and models, we analyse the quality of their predictions and uncertainties in a variety of tasks (binary, regression) and datasets. We also introduce two simulated experiments that evaluate their performance: (1) Bayesian optimization guided molecular design, (2) inference on out-of-distribution data via ablated cluster splits. We offer practical insights into model and feature choice for modelling small chemical datasets, a common scenario in new chemical experiments. We have packaged our analysis into the DIONYSUS repository, which is open sourced to aid in reproducibility and extension to new datasets.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 35 of its 96 references that are in the map. First posted on arXiv 3 Dec 2022.

Builds on

35 of 96 references.

References · 35

  1. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,120
  2. 2017On Calibration of Modern Neural Networkscited by 4,860
  3. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  4. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  5. 2017Neural Message Passing for Quantum Chemistrycited by 3,460
  6. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,841
  7. 2018Relational inductive biases, deep learning, and graph networkscited by 1,341
  8. 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,274
  9. 2017MoleculeNet: A Benchmark for Molecular Machine Learningcited by 1,037
  10. 2018Evidential Deep Learning to Quantify Classification Uncertaintycited by 728
  11. 2015Variational Dropout and the Local Reparameterization Trickcited by 489
  12. 2015Deep Kernel Learningcited by 440
23 more
  1. 2020Simple and Principled Uncertainty Estimation with Deterministic Deep Learning via Distance Awarenesscited by 389
  2. 2019Deep Evidential Regressioncited by 354
  3. 2016GPflow: A Gaussian process library using TensorFlowcited by 139
  4. 2016Bayesian Optimization with Safety Constraints: Safe and Automatic Parameter Tuning in Roboticscited by 134
  5. 2019Variational Bayesian Optimal Experimental Designcited by 116
  6. 2019NGBoost: Natural Gradient Boosting for Probabilistic Predictioncited by 94
  7. 2016Low Data Drug Discovery with One-shot Learningcited by 91
  8. 2021Uncertainty Baselines: Benchmarks for Uncertainty & Robustness in Deep Learningcited by 81
  9. 2020Uncertainty Quantification Using Neural Networks for Molecular Property Predictioncited by 66
  10. 2020Accelerating high-throughput virtual screening through molecular pool-based active learningcited by 59
  11. 2020A Framework for Interdomain and Multioutput Gaussian Processescited by 47
  12. 2017A Network Perspective on Stratification of Multi-Label Datacited by 37
  13. 2020Calibrated Reliable Regression using Maximum Mean Discrepancycited by 36
  14. 2020Gryffin: An algorithm for Bayesian optimization of categorical variables informed by expert knowledgecited by 30
  15. 2019Machine Learning for Scent: Learning Generalizable Perceptual Representations of Small Moleculescited by 28
  16. 2021Reliable Graph Neural Networks for Drug Discovery Under Distributional Shiftcited by 22
  17. 2019Bayesian semi-supervised learning for uncertainty-calibrated prediction of molecular properties and active learningcited by 21
  18. 2021Calibrated Uncertainty for Molecular Property Prediction using Ensembles of Message Passing Neural Networkscited by 21
  19. 2020Gaussian Process Molecule Property Prediction with FlowMOcited by 18
  20. 2021Assigning Confidence to Molecular Property Predictioncited by 14
  21. 2022Self-focusing virtual screening with active design space pruningcited by 8
  22. 2019Deploying a Top-100 Supercomputer for Large Parallel Workloads: the Niagara Supercomputercited by 6
  23. 2019Reliable Prediction Errors for Deep Neural Networks Using Test-Time Dropoutcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 3; 2024: 3; 2025: 1; 2026: 2.

10 papers.

  1. 2024Position: Bayesian Deep Learning is Needed in the Age of Large-Scale AIcited by 60
  2. 2025El Agente: An Autonomous Agent for Quantum Chemistrycited by 43
  3. 2022GAUCHE: A Library for Gaussian Processes in Chemistrycited by 40
  4. 2023Improving Self-supervised Molecular Representation Learning using Persistent Homologycited by 6
  5. 2024The fast committor machine: Interpretable prediction with kernelscited by 5
  6. 2024Ranking over Regression for Bayesian Optimization and Molecule Selectioncited by 4
  7. 2023DOMINO++: Domain-aware Loss Regularization for Deep Learning Generalizabilitycited by 3
  8. 2026Frugal Bayesian Optimization: Scalable Surrogates for Data- and Resource-Limited Discoverycited by 1
  9. 2023Applications of Gaussian Processes at Extreme Lengthscales: From Molecules to Black Holescited by 0
  10. 2026Benchmarking Peptide-Protein Affinity Prediction Across Peptide and Target Shiftscited by 0