An Exploratory Study of AI System Risk Assessment from the Lens of Data Distribution and Uncertainty
Not on the map. Why?
Abstract (arXiv)
Deep learning (DL) has become a driving force and has been widely adopted in many domains and applications with competitive performance. In practice, to solve the nontrivial and complicated tasks in real-world applications, DL is often not used standalone, but instead contributes as a piece of gadget of a larger complex AI system. Although there comes a fast increasing trend to study the quality issues of deep neural networks (DNNs) at the model level, few studies have been performed to investigate the quality of DNNs at both the unit level and the potential impacts on the system level. More importantly, it also lacks systematic investigation on how to perform the risk assessment for AI systems from unit level to system level. To bridge this gap, this paper initiates an early exploratory study of AI system risk assessment from both the data distribution and uncertainty angles to address these issues. We propose a general framework with an exploratory study for analyzing AI systems. After large-scale (700+ experimental configurations and 5000+ GPU hours) experiments and in-depth investigations, we reached a few key interesting findings that highlight the practical need and opportunities for more in-depth investigations into AI systems.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 67 of its 98 references that are in the map. First posted on arXiv 13 Dec 2022.
Builds on
67 of 98 references.
References · 67
- 2014Explaining and Harnessing Adversarial Examplescited by 7,364
- 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,448
- 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,926
- 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
- 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,877
- 2018YOLOv3: An Incremental Improvementcited by 2,659
- 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,385
- 2016Concrete Problems in AI Safetycited by 1,642
- 2018A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attackscited by 1,485
- 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,384
- 2020WILDS: A Benchmark of in-the-Wild Distribution Shiftscited by 1,375
- 2017Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networkscited by 1,328
38 more
- 2020Energy-based Out-of-distribution Detectioncited by 1,299
- 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,287
- 2019Objects as Pointscited by 1,175
- 2021YOLOX: Exceeding YOLO Series in 2021cited by 1,071
- 2016End to End Learning for Self-Driving Carscited by 1,009
- 2018CosFace: Large Margin Cosine Loss for Deep Face Recognitioncited by 971
- 2018Deep Anomaly Detection with Outlier Exposurecited by 942
- 2017Simple Online and Realtime Tracking with a Deep Association Metriccited by 910
- 2016Simple Online and Realtime Trackingcited by 868
- 2021ByteTrack: Multi-Object Tracking by Associating Every Detection Boxcited by 823
- 2020A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challengescited by 819
- 2018Evidential Deep Learning to Quantify Classification Uncertaintycited by 732
- 2021A Survey of Uncertainty in Deep Neural Networkscited by 680
- 2018Predictive Uncertainty Estimation via Prior Networkscited by 557
- 2016MOT16: A Benchmark for Multi-Object Trackingcited by 532
- 2015Deep Speech 2: End-to-End Speech Recognition in English and Mandarincited by 451
- 2020Generalized ODIN: Detecting Out-of-distribution Image without Learning from Out-of-distribution Datacited by 408
- 2017Training Confidence-calibrated Classifiers for Detecting Out-of-Distribution Samplescited by 389
- 2014Deep Speech: Scaling up end-to-end speech recognitioncited by 354
- 2019SelectiveNet: A Deep Neural Network with an Integrated Reject Optioncited by 350
- 2017DeepXplore: Automated Whitebox Testing of Deep Learning Systemscited by 311
- 2018Aleatoric uncertainty estimation with test-time augmentation for medical image segmentation with convolutional neural networkscited by 280
- 2016A Network-based End-to-End Trainable Task-oriented Dialogue Systemcited by 252
- 2017DeepTest: Automated Testing of Deep-Neural-Network-driven Autonomous Carscited by 251
- 2020MultiWOZ 2.2 : A Dialogue Dataset with Additional Annotation Corrections and State Tracking Baselinescited by 204
- 2019Evaluating Scalable Bayesian Deep Learning Methods for Robust Computer Visioncited by 182
- 2015Bayesian Convolutional Neural Networks with Bernoulli Approximate Variational Inferencecited by 159
- 2018DeepGauge: Multi-Granularity Testing Criteria for Deep Learning Systemscited by 147
- 2015Semantically Conditioned LSTM-based Natural Language Generation for Spoken Dialogue Systemscited by 134
- 2019Self-Training for End-to-End Speech Recognitioncited by 129
- 2018Guiding Deep Learning System Testing using Surprise Adequacycited by 124
- 2019End-to-end ASR: from Supervised to Semi-Supervised Learning with Modern Architecturescited by 120
- 2020Rethinking the competition between detection and ReID in Multi-Object Trackingcited by 101
- 2018Towards Safe Autonomous Driving: Capture Uncertainty in the Deep Neural Network For Lidar 3D Vehicle Detectioncited by 82
- 2021Provable Guarantees for Understanding Out-of-distribution Detectioncited by 78
- 2018DeepMutation: Mutation Testing of Deep Learning Systemscited by 67
- 2018Direct Uncertainty Prediction for Medical Second Opinionscited by 61
- 2017End-to-End Task-Completion Neural Dialogue Systemscited by 52
50 most cited of 67 All 67 (JSON)
Built on by
Citing papers per year: 2023: 1; 2024: 1; 2025: 0; 2026: 1.
3 papers.