# Towards Analyzing Semantic Robustness of Deep Neural Networks

arXiv: https://arxiv.org/abs/1904.04621 · HTML: https://mlsanity.com/paper/1904.04621/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Apr 2019.

## Built on by (most cited first, 9 of 9)

- [MVImgNet: A Large-scale Dataset of Multi-view Images](https://mlsanity.com/paper/2303.06042/) (2023)
- [Dataset Interfaces: Diagnosing Model Failures Using Controllable Counterfactual Generation](https://mlsanity.com/paper/2302.07865/) (2023)
- [3DB: A Framework for Debugging Computer Vision Models](https://mlsanity.com/paper/2106.03805/) (2021)
- [Towards Viewpoint-Invariant Visual Recognition via Adversarial Training](https://mlsanity.com/paper/2307.10235/) (2023)
- [DeepCert: Verification of Contextually Relevant Robustness for Neural Network Image Classifiers](https://mlsanity.com/paper/2103.01629/) (2021)
- [Omniview-Tuning: Boosting Viewpoint Invariance of Vision-Language Pre-training Models](https://mlsanity.com/paper/2404.12139/) (2024)
- [Improving Viewpoint Robustness for Visual Recognition via Adversarial Training](https://mlsanity.com/paper/2307.11528/) (2023)
- [AdvDreamer Unveils: Are Vision-Language Models Truly Ready for Real-World 3D Variations?](https://mlsanity.com/paper/2412.03002/) (2024)
- [A Survey of Neural Network Robustness Assessment in Image Recognition](https://mlsanity.com/paper/2404.08285/) (2024)

## Abstract (arXiv)

Despite the impressive performance of Deep Neural Networks (DNNs) on various vision tasks, they still exhibit erroneous high sensitivity toward semantic primitives (e.g. object pose). We propose a theoretically grounded analysis for DNN robustness in the semantic space. We qualitatively analyze different DNNs' semantic robustness by visualizing the DNN global behavior as semantic maps and observe interesting behavior of some DNNs. Since generating these semantic maps does not scale well with the dimensionality of the semantic space, we develop a bottom-up approach to detect robust regions of DNNs. To achieve this, we formalize the problem of finding robust semantic regions of the network as optimizing integral bounds and we develop expressions for update directions of the region bounds. We use our developed formulations to quantitatively evaluate the semantic robustness of different popular network architectures. We show through extensive experimentation that several networks, while trained on the same dataset and enjoying comparable accuracy, do not necessarily perform similarly in semantic robustness. For example, InceptionV3 is more accurate despite being less semantically robust than ResNet50. We hope that this tool will serve as a milestone towards understanding the semantic robustness of DNNs.