mlsanityData to 7 Oct 2026

Adversarial Examples, Uncertainty, and Transfer Testing Robustness in Gaussian Process Hybrid Deep Networks

arXiv 1707.02476 ↗First posted 8 Jul 2017stat.MLcited by 48

Not on the map. Why?

Abstract (arXiv)

Deep neural networks (DNNs) have excellent representative power and are state of the art classifiers on many tasks. However, they often do not capture their own uncertainties well making them less robust in the real world as they overconfidently extrapolate and do not notice domain shift. Gaussian processes (GPs) with RBF kernels on the other hand have better calibrated uncertainties and do not overconfidently extrapolate far from data in their training set. However, GPs have poor representational power and do not perform as well as DNNs on complex domains. In this paper we show that GP hybrid deep networks, GPDNNs, (GPs on top of DNNs and trained end-to-end) inherit the nice properties of both GPs and DNNs and are much more robust to adversarial examples. When extrapolating to adversarial examples and testing in domain shift settings, GPDNNs frequently output high entropy class probabilities corresponding to essentially "don't know". GPDNNs are therefore promising as deep architectures that know when they don't know.

Key facts. Cited by 48 papers in the map (data through 7 Oct 2026). First posted on arXiv 8 Jul 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 18; 2022: 11; 2023: 11; 2024: 2; 2025: 4; 2026: 2.

48 papers.

Sort:

  1. 2021Priors in Bayesian Deep Learning: A Reviewcited by 101
  2. 2021On Feature Collapse and Deep Kernel Learning for Single Forward Pass Uncertaintycited by 87
  3. 2021The Promises and Pitfalls of Deep Kernel Learningcited by 81
  4. 2022A Simple Approach to Improve Single-Model Deep Uncertainty via Distance-Awarenesscited by 49
  5. 2023Quantification of Uncertainty with Adversarial Modelscited by 28
  6. 2023Introducing an Improved Information-Theoretic Measure of Predictive Uncertaintycited by 28
  7. 2021GP-Tree: A Gaussian Process Classifier for Few-Shot Incremental Learningcited by 26
  8. 2021Last Layer Marginal Likelihood for Invariance Learningcited by 21
  9. 2022Meta-learning Adaptive Deep Kernel Gaussian Processes for Molecular Property Predictioncited by 21
  10. 2023Calibrating Transformers via Sparse Gaussian Processescited by 21
  11. 2021Sparse Uncertainty Representation in Deep Learning with Inducing Weightscited by 16
  12. 2024On Information-Theoretic Measures of Predictive Uncertaintycited by 16
36 more
  1. 2023Mitigating Transformer Overconfidence via Lipschitz Regularizationcited by 15
  2. 2021DeepAdversaries: Examining the Robustness of Deep Learning Models for Galaxy Morphology Classificationcited by 10
  3. 2022Uncertainty Estimation for Multi-view Data: The Power of Seeing the Whole Picturecited by 10
  4. 2023A Hierarchical Spatial Transformer for Massive Point Samples in Continuous Spacecited by 10
  5. 2025Revisiting Kernel Attention with Correlated Gaussian Process Representationcited by 9
  6. 2022Interpretable Uncertainty Quantification in AI for HEPcited by 8
  7. 2023Probabilistic Circuits That Know What They Don't Knowcited by 7
  8. 2021Key Technology Considerations in Developing and Deploying Machine Learning Models in Clinical Radiology Practicecited by 6
  9. 2021Towards Adversarial Patch Analysis and Certified Defense against Crowd Countingcited by 6
  10. 2022Deep Kernel Learning of Dynamical Models from High-Dimensional Noisy Datacited by 6
  11. 2023Guided Deep Kernel Learningcited by 6
  12. 2021Adversarial Robustness Guarantees for Gaussian Processescited by 5
  13. 2021Stochastic-Shield: A Probabilistic Approach Towards Training-Free Adversarial Defense in Quantized CNNscited by 5
  14. 2021Uncertainty-Aware Time-to-Event Prediction using Deep Kernel Accelerated Failure Time Modelscited by 5
  15. 2021Adversarial Attack for Uncertainty Estimation: Identifying Critical Regions in Neural Networkscited by 4
  16. 2022On Attacking Out-Domain Uncertainty Estimation in Deep Neural Networkscited by 4
  17. 2021Calibrated simplex-mapping classificationcited by 3
  18. 2021Quantifying Predictive Uncertainty in Medical Image Analysis with Deep Kernel Learningcited by 3
  19. 2023Convolutional Deep Kernel Machinescited by 3
  20. 2024Uncertainty Quantification on Graph Learning: A Surveycited by 3
  21. 2022Unsupervised Restoration of Weather-affected Images using Deep Gaussian Process-based CycleGANcited by 2
  22. 2022B'ezier Gaussian Processes for Tall and Wide Datacited by 2
  23. 2023Evaluation of machine learning architectures on the quantification of epistemic and aleatoric uncertainties in complex dynamical systemscited by 2
  24. 2025From Deep Additive Kernel Learning to Last-Layer Bayesian Neural Networks via Induced Prior Approximationcited by 2
  25. 2021A Modulation Layer to Increase Neural Network Robustness Against Data Quality Issuescited by 1
  26. 2022Distributional Gaussian Processes Layers for Out-of-Distribution Detectioncited by 1
  27. 2022Success of Uncertainty-Aware Deep Models Depends on Data Manifold Geometrycited by 1
  28. 2022Light curve completion and forecasting using fast and scalable Gaussian processes (MuyGPs)cited by 1
  29. 2021Highly Efficient Representation and Active Learning Framework and Its Application to Imbalanced Medical Image Classificationcited by 0
  30. 2021Deep Bayesian Image Set Classification: A Defence Approach against Adversarial Attackscited by 0
  31. 2023Modeling groundwater levels in California's Central Valley by hierarchical Gaussian process and neural network regressioncited by 0
  32. 2023A Kronecker product accelerated efficient sparse Gaussian Process (E-SGP) for flow emulationcited by 0
  33. 2025Scalable Deep Basis Kernel Gaussian Processescited by 0
  34. 2025Response to Promises and Pitfalls of Deep Kernel Learningcited by 0
  35. 2026Diffusion-Inspired Reconfiguration of Transformers for Uncertainty Calibrationcited by 0
  36. 2026Bayesian Scattering: A Principled Baseline for Uncertainty on Image Datacited by 0