mlsanityData to 7 Oct 2026

3DB: A Framework for Debugging Computer Vision Models

arXiv 2106.03805 ↗First posted 7 Jun 2021cs.CV cs.LG stat.MLcited by 31

Not on the map. Why?

Abstract (arXiv)

We introduce 3DB: an extendable, unified framework for testing and debugging vision models using photorealistic simulation. We demonstrate, through a wide range of use cases, that 3DB allows users to discover vulnerabilities in computer vision systems and gain insights into how models make decisions. 3DB captures and generalizes many robustness analyses from prior work, and enables one to study their interplay. Finally, we find that the insights generated by the system transfer to the physical world. We are releasing 3DB as a library (https://github.com/3db/3db) alongside a set of example analyses, guides, and documentation: https://3db.github.io/3db/ .

Key facts. Cited by 31 papers in the map (data through 7 Oct 2026). Builds on 63 of its 95 references that are in the map. First posted on arXiv 7 Jun 2021.

Builds on

63 of 95 references.

References · 63

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  3. 2013Intriguing properties of neural networkscited by 5,114
  4. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  5. 2017Axiomatic Attribution for Deep Networkscited by 3,350
  6. 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
  7. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,348
  8. 2016OpenAI Gymcited by 1,684
  9. 2019Do ImageNet Classifiers Generalize to ImageNet?cited by 1,507
  10. 2018ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustnesscited by 1,430
  11. 2019Natural Adversarial Examplescited by 1,401
  12. 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,309
38 more
  1. 2019Habitat: A Platform for Embodied AI Researchcited by 1,163
  2. 2018Sanity Checks for Saliency Mapscited by 1,149
  3. 2017AI2-THOR: An Interactive 3D Environment for Visual AIcited by 850
  4. 2017Network Dissection: Quantifying Interpretability of Deep Visual Representationscited by 703
  5. 2017Interpretable Explanations of Black Boxes by Meaningful Perturbationcited by 611
  6. 2020Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understandingcited by 593
  7. 2020SAPIEN: A SimulAted Part-based Interactive ENvironmentcited by 584
  8. 2017AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehiclescited by 517
  9. 2017Adversarial Patchcited by 514
  10. 2020robosuite: A Modular Simulation Framework and Benchmark for Robot Learningcited by 484
  11. 2018Gibson Env: Real-World Perception for Embodied Agentscited by 477
  12. 2020Measuring Robustness to Natural Distribution Shifts in Image Classificationcited by 452
  13. 2017Interpretation of Neural Networks is Fragilecited by 443
  14. 2017Neural 3D Mesh Renderercited by 437
  15. 2018VirtualHome: Simulating Household Activities via Programscited by 411
  16. 2020Noise or Signal: The Role of Image Backgrounds in Object Recognitioncited by 342
  17. 2017Adversarial Attacks on Neural Network Policiescited by 314
  18. 2019Counterfactual Visual Explanationscited by 312
  19. 2018On the Robustness of Interpretability Methodscited by 310
  20. 2017Adversarial Examples for Semantic Segmentation and Object Detectioncited by 275
  21. 2017Real Time Image Saliency for Black Box Classifierscited by 246
  22. 2019On Completeness-aware Concept-Based Explanations in Deep Neural Networkscited by 239
  23. 2018Unity: A General Platform for Intelligent Agentscited by 232
  24. 2018Physical Adversarial Examples for Object Detectorscited by 163
  25. 2018DPatch: An Adversarial Patch Attack on Object Detectorscited by 146
  26. 2020Debugging Tests for Model Explanationscited by 146
  27. 2018Strike (with) a Pose: Neural Networks Are Easily Fooled by Strange Poses of Familiar Objectscited by 143
  28. 2019Meta-Sim: Learning to Generate Synthetic Datasetscited by 128
  29. 2019Adversarial Examples Are a Natural Consequence of Test Error in Noisecited by 114
  30. 2018Building Generalizable Agents with a Realistic and Rich 3D Environmentcited by 109
  31. 2018Benchmarking Neural Network Robustness to Common Corruptions and Surface Variationscited by 99
  32. 2015Benchmarking in Manipulation Research: The YCB Object and Model Set and Benchmarking Protocolscited by 98
  33. 2019Interactive Gibson Benchmark (iGibson 0.5): A Benchmark for Interactive Navigation in Cluttered Environmentscited by 89
  34. 2021Leveraging Sparse Linear Layers for Debuggable Deep Networkscited by 82
  35. 2019BlenderProccited by 76
  36. 2016DeepMind Labcited by 74
  37. 2019Adversarial camera stickers: A physical camera-based attack on deep learning systemscited by 69
  38. 2020Flightmare: A Flexible Quadrotor Simulatorcited by 69

50 most cited of 63 All 63 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 3; 2022: 13; 2023: 9; 2024: 3; 2025: 2; 2026: 1.

31 papers.

Sort:

By area:

  1. 2022Datamodels: Predicting Predictions from Training Datacited by 186
  2. 2022Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networkscited by 126
  3. 20223D Common Corruptions and Data Augmentationcited by 120
  4. 2021Salient ImageNet: How to discover spurious features in Deep Learning?cited by 114
  5. 2022A Whac-A-Mole Dilemma: Shortcuts Come in Multiples Where Mitigating One Amplifies Otherscited by 85
  6. 2023Diagnosing and Rectifying Vision Models using Languagecited by 58
  7. 2022Adaptive Testing of Computer Vision Modelscited by 39
  8. 2022Missingness Bias in Model Debuggingcited by 36
  9. 2023Dataset Interfaces: Diagnosing Model Failures Using Controllable Counterfactual Generationcited by 35
  10. 2022ModelDiff: A Framework for Comparing Learning Algorithmscited by 30
  11. 2021Robust Feature-Level Adversaries are Interpretability Toolscited by 27
  12. 2022When does Bias Transfer in Transfer Learning?cited by 23
19 more
  1. 2023Red Teaming Deep Neural Networks with Feature Synthesis Toolscited by 21
  2. 2022REAP: A Large-Scale Realistic Adversarial Patch Benchmarkcited by 17
  3. 2023Rethinking Backdoor Attackscited by 14
  4. 2023OOD-CV-v2: An extended Benchmark for Robustness to Out-of-Distribution Shifts of Individual Nuisances in Natural Imagescited by 12
  5. 2023DiG-IN: Diffusion Guidance for Investigating Networks -- Uncovering Classifier Differences Neuron Visualisations and Visual Counterfactual Explanationscited by 11
  6. 2024Trapped in texture bias? A large scale comparison of deep instance segmentationcited by 10
  7. 2023PRIME: Prioritizing Interpretability in Failure Mode Extractioncited by 9
  8. 2021FOCUS: Familiar Objects in Common and Uncommon Settingscited by 8
  9. 2023Knowledge-based in silico models and dataset for the comparative evaluation of mammography AI for a range of breast characteristics, lesion conspicuities and dosescited by 8
  10. 2022Diagnostics for Deep Neural Networks with Automated Copy/Paste Attackscited by 6
  11. 2022Towards a More Rigorous Science of Blindspot Discovery in Image Classification Modelscited by 5
  12. 2023Identifying Systematic Errors in Object Detectors with the SCROD Pipelinecited by 4
  13. 2022Causal Scene BERT: Improving object detection by searching for challenging groups of datacited by 3
  14. 2022Data-Centric Debugging: mitigating model failures via targeted data collectioncited by 3
  15. 2024RenderBender: A Survey on Adversarial Attacks Using Differentiable Renderingcited by 3
  16. 2025BiMa: Towards Biases Mitigation for Text-Video Retrieval via Scene Element Guidancecited by 2
  17. 2024Identification of Fine-grained Systematic Errors via Controlled Scene Generationcited by 0
  18. 2025SafeFix: Targeted Model Repair via Controlled Image Generationcited by 0
  19. 2026Auditing Quality Filters for Long-Tail Human Data Curationcited by 0