# Surrogate Model-Based Explainability Methods for Point Cloud NNs

arXiv: https://arxiv.org/abs/2107.13459 · HTML: https://mlsanity.com/paper/2107.13459/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 42 references that are in the map.
- First posted on arXiv 28 Jul 2021.

## Builds on (most cited first, 29 of 29)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI](https://mlsanity.com/paper/1910.10045/) (2019)
- [Learning Important Features Through Propagating Activation Differences](https://mlsanity.com/paper/1704.02685/) (2017)
- [Sanity Checks for Saliency Maps](https://mlsanity.com/paper/1810.03292/) (2018)
- [SmoothGrad: removing noise by adding noise](https://mlsanity.com/paper/1706.03825/) (2017)
- [Spectral Networks and Locally Connected Networks on Graphs](https://mlsanity.com/paper/1312.6203/) (2013)
- [Striving for Simplicity: The All Convolutional Net](https://mlsanity.com/paper/1412.6806/) (2014)
- [Frustum PointNets for 3D Object Detection from RGB-D Data](https://mlsanity.com/paper/1711.08488/) (2017)
- [Evaluating the visualization of what a Deep Neural Network has learned](https://mlsanity.com/paper/1509.06321/) (2015)
- [Volumetric and Multi-View CNNs for Object Classification on 3D Data](https://mlsanity.com/paper/1604.03265/) (2016)
- [A Survey on the Explainability of Supervised Machine Learning](https://mlsanity.com/paper/2011.07876/) (2020)
- [PointCloud Saliency Maps](https://mlsanity.com/paper/1812.01687/) (2018)
- [L-Shapley and C-Shapley: Efficient Model Interpretation for Structured Data](https://mlsanity.com/paper/1808.02610/) (2018)
- [Understanding Neural Networks via Feature Visualization: A survey](https://mlsanity.com/paper/1904.08939/) (2019)
- [PointHop: An Explainable Machine Learning Method for Point Cloud Classification](https://mlsanity.com/paper/1907.12766/) (2019)
- [Why are Saliency Maps Noisy? Cause of and Solution to Noisy Saliency Maps](https://mlsanity.com/paper/1902.04893/) (2019)
- [Gradients of Counterfactuals](https://mlsanity.com/paper/1611.02639/) (2016)
- [Detecting Unexpected Obstacles for Self-Driving Cars: Fusing Deep Learning and Geometric Modeling](https://mlsanity.com/paper/1612.06573/) (2016)
- [Explaining Autonomous Driving by Learning End-to-End Visual Attention](https://mlsanity.com/paper/2006.03347/) (2020)
- [Deep neural network or dermatologist?](https://mlsanity.com/paper/1908.06612/) (2019)
- [3D Point Cloud Feature Explanations Using Gradient-Based Methods](https://mlsanity.com/paper/2006.05548/) (2020)
- [Study of Different Deep Learning Approach with Explainable AI for Screening Patients with COVID-19 Symptoms: Using CT Scan and Chest X-ray Image Dataset](https://mlsanity.com/paper/2007.12525/) (2020)
- [Evaluating the performance of the LIME and Grad-CAM explanation methods on a LEGO multi-label image classification task](https://mlsanity.com/paper/2008.01584/) (2020)

## Built on by (most cited first, 18 of 18)

- [Contestable AI needs Computational Argumentation](https://mlsanity.com/paper/2405.10729/) (2024)
- [Visualizing Global Explanations of Point Cloud DNNs](https://mlsanity.com/paper/2203.09505/) (2022)
- [Explainability-Aware One Point Attack for Point Cloud Neural Networks](https://mlsanity.com/paper/2110.04158/) (2021)
- [Fast and Simple Explainability for Point Cloud Networks](https://mlsanity.com/paper/2403.07706/) (2024)
- [Explainability of Point Cloud Neural Networks Using SMILE: Statistical Model-Agnostic Interpretability with Local Explanations](https://mlsanity.com/paper/2410.15374/) (2024)
- [Maximum Entropy Baseline for Integrated Gradients](https://mlsanity.com/paper/2204.05948/) (2022)
- [One Wave To Explain Them All: A Unifying Perspective On Feature Attribution](https://mlsanity.com/paper/2410.01482/) (2024)
- [Concept Activation Vectors for Generating User-Defined 3D Shapes](https://mlsanity.com/paper/2205.02102/) (2022)
- [Robustifying Point Cloud Networks by Refocusing](https://mlsanity.com/paper/2308.05525/) (2023)
- [InfoCons: Identifying Interpretable Critical Concepts in Point Clouds via Information Theory](https://mlsanity.com/paper/2505.19820/) (2025)
- [DAM: Diffusion Activation Maximization for 3D Global Explanations](https://mlsanity.com/paper/2401.14938/) (2024)
- [Flow AM: Generating Point Cloud Global Explanations by Latent Alignment](https://mlsanity.com/paper/2404.18760/) (2024)
- [Explainable LiDAR 3D Point Cloud Segmentation and Clustering for Detecting Airplane-Generated Wind Turbulence](https://mlsanity.com/paper/2503.00518/) (2025)
- [XAI for Point Cloud Data using Perturbations based on Meaningful Segmentation](https://mlsanity.com/paper/2507.22020/) (2025)
- [Explaining Black-box Model Predictions via Two-level Nested Feature Attributions with Consistency Property](https://mlsanity.com/paper/2405.14522/) (2024)
- [PointExplainer: Towards Transparent Parkinson's Disease Diagnosis](https://mlsanity.com/paper/2505.03833/) (2025)
- [XSPLAIN: XAI-enabling Splat-based Prototype Learning for Attribute-aware INterpretability](https://mlsanity.com/paper/2602.10239/) (2026)
- [Synthetic LiDAR Data Generation and Deterministic Downsampling for Point Cloud Classification on the Edge](https://mlsanity.com/paper/2608.07106/) (2026)

## Abstract (arXiv)

In the field of autonomous driving and robotics, point clouds are showing their excellent real-time performance as raw data from most of the mainstream 3D sensors. Therefore, point cloud neural networks have become a popular research direction in recent years. So far, however, there has been little discussion about the explainability of deep neural networks for point clouds. In this paper, we propose a point cloud-applicable explainability approach based on local surrogate model-based method to show which components contribute to the classification. Moreover, we propose quantitative fidelity validations for generated explanations that enhance the persuasive power of explainability and compare the plausibility of different existing point cloud-applicable explainability methods. Our new explainability approach provides a fairly accurate, more semantically coherent and widely applicable explanation for point cloud classification tasks. Our code is available at https://github.com/Explain3D/LIME-3D