# Semantic Segmentation for Real Point Cloud Scenes via Bilateral Augmentation and Adaptive Fusion

arXiv: https://arxiv.org/abs/2103.07074 · HTML: https://mlsanity.com/paper/2103.07074/

## Key facts

- Cited by 72 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 57 references that are in the map.
- First posted on arXiv 12 Mar 2021.

## Builds on (most cited first, 30 of 35)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Pyramid Scene Parsing Network](https://mlsanity.com/paper/1612.01105/) (2016)
- [Dynamic Graph CNN for Learning on Point Clouds](https://mlsanity.com/paper/1801.07829/) (2018)
- [SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation](https://mlsanity.com/paper/1511.00561/) (2015)
- [4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks](https://mlsanity.com/paper/1904.08755/) (2019)
- [KPConv: Flexible and Deformable Convolution for Point Clouds](https://mlsanity.com/paper/1904.08889/) (2019)
- [PointCNN: Convolution On X-Transformed Points](https://mlsanity.com/paper/1801.07791/) (2018)
- [PointConv: Deep Convolutional Networks on 3D Point Clouds](https://mlsanity.com/paper/1811.07246/) (2018)
- [RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Clouds](https://mlsanity.com/paper/1911.11236/) (2019)
- [Deep Learning for 3D Point Clouds: A Survey](https://mlsanity.com/paper/1912.12033/) (2019)
- [PointContrast: Unsupervised Pre-training for 3D Point Cloud Understanding](https://mlsanity.com/paper/2007.10985/) (2020)
- [Joint 2D-3D-Semantic Data for Indoor Scene Understanding](https://mlsanity.com/paper/1702.01105/) (2017)
- [Large-scale Point Cloud Semantic Segmentation with Superpoint Graphs](https://mlsanity.com/paper/1711.09869/) (2017)
- [Relation-Shape Convolutional Neural Network for Point Cloud Analysis](https://mlsanity.com/paper/1904.07601/) (2019)
- [PolarNet: An Improved Grid Representation for Online LiDAR Point Clouds Semantic Segmentation](https://mlsanity.com/paper/2003.14032/) (2020)
- [PointASNL: Robust Point Clouds Processing using Nonlocal Neural Networks with Adaptive Sampling](https://mlsanity.com/paper/2003.00492/) (2020)
- [SqueezeSegV2: Improved Model Structure and Unsupervised Domain Adaptation for Road-Object Segmentation from a LiDAR Point Cloud](https://mlsanity.com/paper/1809.08495/) (2018)
- [Tangent Convolutions for Dense Prediction in 3D](https://mlsanity.com/paper/1807.02443/) (2018)
- [PU-GAN: a Point Cloud Upsampling Adversarial Network](https://mlsanity.com/paper/1907.10844/) (2019)
- [Semantic3D.net: A new Large-scale Point Cloud Classification Benchmark](https://mlsanity.com/paper/1704.03847/) (2017)
- [SPLATNet: Sparse Lattice Networks for Point Cloud Processing](https://mlsanity.com/paper/1802.08275/) (2018)
- [Bi-directional Cross-Modality Feature Propagation with Separation-and-Aggregation Gate for RGB-D Semantic Segmentation](https://mlsanity.com/paper/2007.09183/) (2020)
- [ShellNet: Efficient Point Cloud Convolutional Neural Networks using Concentric Shells Statistics](https://mlsanity.com/paper/1908.06295/) (2019)
- [Point2Sequence: Learning the Shape Representation of 3D Point Clouds with an Attention-based Sequence to Sequence Network](https://mlsanity.com/paper/1811.02565/) (2018)

## Built on by (most cited first, 30 of 72)

- [PointNeXt: Revisiting PointNet++ with Improved Training and Scaling Strategies](https://mlsanity.com/paper/2206.04670/) (2022)
- [SSCBench: A Large-Scale 3D Semantic Scene Completion Benchmark for Autonomous Driving](https://mlsanity.com/paper/2306.09001/) (2023)
- [MSeg3D: Multi-modal 3D Semantic Segmentation for Autonomous Driving](https://mlsanity.com/paper/2303.08600/) (2023)
- [Contrastive Boundary Learning for Point Cloud Segmentation](https://mlsanity.com/paper/2203.05272/) (2022)
- [CENet: Toward Concise and Efficient LiDAR Semantic Segmentation for Autonomous Driving](https://mlsanity.com/paper/2207.12691/) (2022)
- [STPLS3D: A Large-Scale Synthetic and Real Aerial Photogrammetry 3D Point Cloud Dataset](https://mlsanity.com/paper/2203.09065/) (2022)
- [Implicit Autoencoder for Point-Cloud Self-Supervised Representation Learning](https://mlsanity.com/paper/2201.00785/) (2022)
- [PVT: Point-Voxel Transformer for Point Cloud Learning](https://mlsanity.com/paper/2108.06076/) (2021)
- [PatchFormer: An Efficient Point Transformer with Patch Attention](https://mlsanity.com/paper/2111.00207/) (2021)
- [Meta Architecture for Point Cloud Analysis](https://mlsanity.com/paper/2211.14462/) (2022)
- [Edge-Cloud Polarization and Collaboration: A Comprehensive Survey for AI](https://mlsanity.com/paper/2111.06061/) (2021)
- [PU-Transformer: Point Cloud Upsampling Transformer](https://mlsanity.com/paper/2111.12242/) (2021)
- [Meta-RangeSeg: LiDAR Sequence Semantic Segmentation Using Multiple Feature Aggregation](https://mlsanity.com/paper/2202.13377/) (2022)
- [Clustering based Point Cloud Representation Learning for 3D Analysis](https://mlsanity.com/paper/2307.14605/) (2023)
- [Cascaded Cross MLP-Mixer GANs for Cross-View Image Translation](https://mlsanity.com/paper/2110.10183/) (2021)
- [VIBUS: Data-efficient 3D Scene Parsing with VIewpoint Bottleneck and Uncertainty-Spectrum Modeling](https://mlsanity.com/paper/2210.11472/) (2022)
- [P2C: Self-Supervised Point Cloud Completion from Single Partial Clouds](https://mlsanity.com/paper/2307.14726/) (2023)
- [Visual Foundation Models Boost Cross-Modal Unsupervised Domain Adaptation for 3D Semantic Segmentation](https://mlsanity.com/paper/2403.10001/) (2024)
- [SVQNet: Sparse Voxel-Adjacent Query Network for 4D Spatio-Temporal LiDAR Semantic Segmentation](https://mlsanity.com/paper/2308.13323/) (2023)
- [Weakly Supervised 3D Point Cloud Segmentation via Multi-Prototype Learning](https://mlsanity.com/paper/2205.03137/) (2022)
- [TFNet: Exploiting Temporal Cues for Fast and Accurate LiDAR Semantic Segmentation](https://mlsanity.com/paper/2309.07849/) (2023)
- [SemAffiNet: Semantic-Affine Transformation for Point Cloud Segmentation](https://mlsanity.com/paper/2205.13490/) (2022)
- [PointCaM: Cut-and-Mix for Open-Set Point Cloud Learning](https://mlsanity.com/paper/2212.02011/) (2022)
- [Investigating Attention Mechanism in 3D Point Cloud Object Detection](https://mlsanity.com/paper/2108.00620/) (2021)
- [PnP-3D: A Plug-and-Play for 3D Point Clouds](https://mlsanity.com/paper/2108.07378/) (2021)
- [Energy-Based Residual Latent Transport for Unsupervised Point Cloud Completion](https://mlsanity.com/paper/2211.06820/) (2022)
- [PerLA: Perceptive 3D Language Assistant](https://mlsanity.com/paper/2411.19774/) (2024)
- [SNE-RoadSegV2: Advancing Heterogeneous Feature Fusion and Fallibility Awareness for Freespace Detection](https://mlsanity.com/paper/2402.18918/) (2024)
- [Continuous Conditional Random Field Convolution for Point Cloud Segmentation](https://mlsanity.com/paper/2110.06085/) (2021)
- [Efficient Urban-scale Point Clouds Segmentation with BEV Projection](https://mlsanity.com/paper/2109.09074/) (2021)

## Abstract (arXiv)

Given the prominence of current 3D sensors, a fine-grained analysis on the basic point cloud data is worthy of further investigation. Particularly, real point cloud scenes can intuitively capture complex surroundings in the real world, but due to 3D data's raw nature, it is very challenging for machine perception. In this work, we concentrate on the essential visual task, semantic segmentation, for large-scale point cloud data collected in reality. On the one hand, to reduce the ambiguity in nearby points, we augment their local context by fully utilizing both geometric and semantic features in a bilateral structure. On the other hand, we comprehensively interpret the distinctness of the points from multiple resolutions and represent the feature map following an adaptive fusion method at point-level for accurate semantic segmentation. Further, we provide specific ablation studies and intuitive visualizations to validate our key modules. By comparing with state-of-the-art networks on three different benchmarks, we demonstrate the effectiveness of our network.