mlsanityData to 7 Oct 2026

Robustness via Cross-Domain Ensembles

arXiv 2103.10919 ↗First posted 19 Mar 2021cs.CV cs.LGcited by 23

Not on the map. Why?

Abstract (arXiv)

We present a method for making neural network predictions robust to shifts from the training data distribution. The proposed method is based on making predictions via a diverse set of cues (called 'middle domains') and ensembling them into one strong prediction. The premise of the idea is that predictions made via different cues respond differently to a distribution shift, hence one should be able to merge them into one robust final prediction. We perform the merging in a straightforward but principled manner based on the uncertainty associated with each prediction. The evaluations are performed using multiple tasks and datasets (Taskonomy, Replica, ImageNet, CIFAR) under a wide range of adversarial and non-adversarial distribution shifts which demonstrate the proposed method is considerably more robust than its standard learning counterpart, conventional deep ensembles, and several other baselines.

Key facts. Cited by 23 papers in the map (data through 7 Oct 2026). Builds on 44 of its 65 references that are in the map. First posted on arXiv 19 Mar 2021.

Builds on

44 of 65 references.

References · 44

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  3. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  4. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  5. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  6. 2013Intriguing properties of neural networkscited by 5,114
  7. 2017On Calibration of Modern Neural Networkscited by 4,860
  8. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  9. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  10. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,857
  11. 2017What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?cited by 2,335
  12. 2018ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustnesscited by 1,430
32 more
  1. 2017Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networkscited by 1,325
  2. 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,274
  3. 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,259
  4. 2019Habitat: A Platform for Embodied AI Researchcited by 1,163
  5. 2019Adversarial Examples Are Not Bugs, They Are Featurescited by 950
  6. 2018Deep Anomaly Detection with Outlier Exposurecited by 939
  7. 2016Adversarial Machine Learning at Scalecited by 934
  8. 2019AugMix: A Simple Data Processing Method to Improve Robustness and Uncertaintycited by 897
  9. 2019The Replica Dataset: A Digital Replica of Indoor Spacescited by 885
  10. 2019On the Convergence of Adam and Beyondcited by 834
  11. 2018Taskonomy: Disentangling Task Transfer Learningcited by 631
  12. 2015Weight Uncertainty in Neural Networkscited by 554
  13. 2018Accurate Uncertainties for Deep Learning Using Calibrated Regressioncited by 417
  14. 2017Training Confidence-calibrated Classifiers for Detecting Out-of-Distribution Samplescited by 389
  15. 2020BatchEnsemble: An Alternative Approach to Efficient Ensemble and Lifelong Learningcited by 368
  16. 2018PAD-Net: Multi-Tasks Guided Prediction-and-Distillation Network for Simultaneous Depth Estimation and Scene Parsingcited by 224
  17. 2019Improving Adversarial Robustness via Promoting Ensemble Diversitycited by 213
  18. 2020Pitfalls of In-Domain Uncertainty Estimation and Ensembling in Deep Learningcited by 208
  19. 2018Deep Bayesian Bandits Showdown: An Empirical Comparison of Bayesian Deep Networks for Thompson Samplingcited by 166
  20. 2017Multiplicative Normalizing Flows for Variational Bayesian Neural Networkscited by 157
  21. 2020Hyperparameter Ensembles for Robustness and Uncertainty Quantificationcited by 149
  22. 2018Flipout: Efficient Pseudo-Independent Weight Perturbations on Mini-Batchescited by 133
  23. 2019Adversarial Examples Are a Natural Consequence of Test Error in Noisecited by 114
  24. 2017A Study and Comparison of Human and Deep Learning Recognition Performance Under Visual Distortionscited by 107
  25. 2020Rethinking Bias-Variance Trade-off for Generalization of Neural Networkscited by 107
  26. 2020Robust Learning Through Cross-Task Consistencycited by 105
  27. 2017Measuring the tendency of CNNs to Learn Surface Statistical Regularitiescited by 87
  28. 2016Structured and Efficient Variational Deep Learning with Matrix Gaussian Posteriorscited by 82
  29. 2020DVERGE: Diversifying Vulnerabilities for Enhanced Robust Generation of Ensemblescited by 73
  30. 2019Improving Adversarial Robustness of Ensembles with Diversity Trainingcited by 69
  31. 2020Learning Loss for Test-Time Augmentationcited by 56
  32. 2020Semi-Supervised Learning for Multi-Task Scene Understanding by Neural Graph Consensuscited by 8

Built on by

Cited by year

Citing papers per year: 2021: 3; 2022: 5; 2023: 11; 2024: 1; 2025: 3.

23 papers.

Sort:

By area:

  1. 2022Diverse Weight Averaging for Out-of-Distribution Generalizationcited by 148
  2. 20223D Common Corruptions and Data Augmentationcited by 120
  3. 2022Auto-Lambda: Disentangling Dynamic Task Relationshipscited by 73
  4. 2022Enhancing Pseudo Label Quality for Semi-Supervised Domain-Generalized Medical Image Segmentationcited by 51
  5. 2023HGFormer: Hierarchical Grouping Transformer for Domain Generalized Semantic Segmentationcited by 31
  6. 2021Test time Adaptation through Perturbation Robustnesscited by 28
  7. 2021Deep Depth from Focus with Differential Focus Volumecited by 20
  8. 2021Combining Diverse Feature Priorscited by 12
  9. 2023HybridAugment++: Unified Frequency Spectra Perturbations for Model Robustnesscited by 12
  10. 2023Rapid Network Adaptation: Learning to Adapt Neural Networks Using Test-Time Feedbackcited by 10
  11. 2023Self-supervised Hypergraphs for Learning Multiple World Interpretationscited by 8
  12. 2023Discretization-Induced Dirichlet Posterior for Robust Uncertainty Quantification on Regressioncited by 7
11 more
  1. 2023Multi-Task Hypergraphs for Semi-supervised Learning using Earth Observationscited by 6
  2. 2023Exploring Model Learning Heterogeneity for Boosting Ensemble Robustnesscited by 6
  3. 2023Adversarial Attacks and Defenses in Machine Learning-Powered Networks: A Contemporary Surveycited by 5
  4. 2022Robust Deep Ensemble Method for Real-world Image Denoisingcited by 4
  5. 2023Uncertainty Voting Ensemble for Imbalanced Deep Regressioncited by 4
  6. 2025Mimir: Hierarchical Goal-Driven Diffusion with Uncertainty Propagation for End-to-End Autonomous Drivingcited by 3
  7. 2023Examining Common Paradigms in Multi-Task Learningcited by 2
  8. 2024WildFit: Autonomous In-situ Model Adaptation for Resource-Constrained IoT Systemscited by 2
  9. 2023Uncertainty estimation in Deep Learning for Panoptic segmentationcited by 1
  10. 2025Pulling Back the Curtain: Unsupervised Adversarial Detection via Contrastive Auxiliary Networkscited by 1
  11. 2025PlantTraitNet: An Uncertainty-Aware Multimodal Framework for Global-Scale Plant Trait Inference from Citizen Science Datacited by 0