# Interpreting Hidden Semantics in the Intermediate Layers of 3D Point Cloud Classification Neural Network

arXiv: https://arxiv.org/abs/2303.06652 · HTML: https://mlsanity.com/paper/2303.06652/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 43 references that are in the map.
- First posted on arXiv 12 Mar 2023.

## Builds on (most cited first, 27 of 27)

- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [Learning Important Features Through Propagating Activation Differences](https://mlsanity.com/paper/1704.02685/) (2017)
- [How transferable are features in deep neural networks?](https://mlsanity.com/paper/1411.1792/) (2014)
- [Striving for Simplicity: The All Convolutional Net](https://mlsanity.com/paper/1412.6806/) (2014)
- [PointConv: Deep Convolutional Networks on 3D Point Clouds](https://mlsanity.com/paper/1811.07246/) (2018)
- [Understanding Deep Image Representations by Inverting Them](https://mlsanity.com/paper/1412.0035/) (2014)
- [A Survey on Neural Network Interpretability](https://mlsanity.com/paper/2012.14261/) (2020)
- [A Survey on the Explainability of Supervised Machine Learning](https://mlsanity.com/paper/2011.07876/) (2020)
- [Interpretable Convolutional Neural Networks](https://mlsanity.com/paper/1710.00935/) (2017)
- [PointCloud Saliency Maps](https://mlsanity.com/paper/1812.01687/) (2018)
- [Interpreting CNNs via Decision Trees](https://mlsanity.com/paper/1802.00121/) (2018)
- [Repurposing GANs for One-shot Semantic Part Segmentation](https://mlsanity.com/paper/2103.04379/) (2021)
- [AdvPC: Transferable Adversarial Perturbations on 3D Point Clouds](https://mlsanity.com/paper/1912.00461/) (2019)
- [Shape-invariant 3D Adversarial Point Clouds](https://mlsanity.com/paper/2203.04041/) (2022)
- [Interpreting CNN Knowledge via an Explanatory Graph](https://mlsanity.com/paper/1708.01785/) (2017)
- [PointGuard: Provably Robust 3D Point Cloud Classification](https://mlsanity.com/paper/2103.03046/) (2021)
- [Minimal Adversarial Examples for Deep Learning on 3D Point Clouds](https://mlsanity.com/paper/2008.12066/) (2020)
- [Imperceptible Transfer Attack and Defense on 3D Point Cloud Classification](https://mlsanity.com/paper/2111.10990/) (2021)
- [Examining CNN Representations with respect to Dataset Bias](https://mlsanity.com/paper/1710.10577/) (2017)
- [Do semantic parts emerge in Convolutional Neural Networks?](https://mlsanity.com/paper/1607.03738/) (2016)
- [Exploring the Devil in Graph Spectral Domain for 3D Point Cloud Attacks](https://mlsanity.com/paper/2202.07261/) (2022)
- [3D Point Cloud Feature Explanations Using Gradient-Based Methods](https://mlsanity.com/paper/2006.05548/) (2020)

## Built on by (most cited first, 1 of 1)

- [Locating and Mitigating Gradient Conflicts in Point Cloud Domain Adaptation via Saliency Map Skewness](https://mlsanity.com/paper/2504.15796/) (2025)

## Abstract (arXiv)

Although 3D point cloud classification neural network models have been widely used, the in-depth interpretation of the activation of the neurons and layers is still a challenge. We propose a novel approach, named Relevance Flow, to interpret the hidden semantics of 3D point cloud classification neural networks. It delivers the class Relevance to the activated neurons in the intermediate layers in a back-propagation manner, and associates the activation of neurons with the input points to visualize the hidden semantics of each layer. Specially, we reveal that the 3D point cloud classification neural network has learned the plane-level and part-level hidden semantics in the intermediate layers, and utilize the normal and IoU to evaluate the consistency of both levels' hidden semantics. Besides, by using the hidden semantics, we generate the adversarial attack samples to attack 3D point cloud classifiers. Experiments show that our proposed method reveals the hidden semantics of the 3D point cloud classification neural network on ModelNet40 and ShapeNet, which can be used for the unsupervised point cloud part segmentation without labels and attacking the 3D point cloud classifiers.