LVIC: Multi-modality segmentation by Lifting Visual Info as Cue
Not on the map. Why?
Abstract (arXiv)
Multi-modality fusion is proven an effective method for 3d perception for autonomous driving. However, most current multi-modality fusion pipelines for LiDAR semantic segmentation have complicated fusion mechanisms. Point painting is a quite straight forward method which directly bind LiDAR points with visual information. Unfortunately, previous point painting like methods suffer from projection error between camera and LiDAR. In our experiments, we find that this projection error is the devil in point painting. As a result of that, we propose a depth aware point painting mechanism, which significantly boosts the multi-modality fusion. Apart from that, we take a deeper look at the desired visual feature for LiDAR to operate semantic segmentation. By Lifting Visual Information as Cue, LVIC ranks 1st on nuScenes LiDAR semantic segmentation benchmark. Our experiments show the robustness and effectiveness. Codes would be make publicly available soon.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 29 of its 35 references that are in the map. First posted on arXiv 8 Mar 2024.
Builds on
29 of 35 references.
References · 29
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
- 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
- 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
- 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
- 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
- 2022A ConvNet for the 2020scited by 3,545
- 20163D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotationcited by 1,471
- 2020Point Transformercited by 1,462
- 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,213
- 2023ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoderscited by 703
- 2023ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depthcited by 646
17 more
- 2023Point Transformer V3: Simpler, Faster, Strongercited by 579
- 2019PointPainting: Sequential Fusion for 3D Object Detectioncited by 449
- 2020Searching Efficient 3D Architectures with Sparse Point-Voxel Convolutioncited by 422
- 2022Point Transformer V2: Grouped Vector Attention and Partition-based Poolingcited by 384
- 2020Exploring Self-attention for Image Recognitioncited by 313
- 2015DeepStereo: Learning to Predict New Views from the World's Imagerycited by 185
- 2021RPVNet: A Deep and Efficient Range-Point-Voxel Fusion Network for LiDAR Point Cloud Segmentationcited by 173
- 2021Multimodal Virtual Point 3D Detectioncited by 166
- 2021Panoptic nuScenes: A Large-Scale Benchmark for LiDAR Panoptic Segmentation and Trackingcited by 165
- 2022LocalBins: Improving Depth Estimation by Learning Local Distributionscited by 92
- 2023MSeg3D: Multi-modal 3D Semantic Segmentation for Autonomous Drivingcited by 68
- 2021Learning 3D Semantic Segmentation with only 2D Image Supervisioncited by 60
- 2021EPMF: Efficient Perception-aware Multi-sensor Fusion for 3D Semantic Segmentationcited by 59
- 2018On the Importance of Stereo for Accurate Depth Estimation: An Efficient Semi-Supervised Deep Neural Network Approachcited by 18
- 2022CPGNet: Cascade Point-Grid Fusion Network for Real-Time LiDAR Semantic Segmentationcited by 15
- 2023PeP: a Point enhanced Painting method for unified point cloud taskscited by 7
- 2023Revisiting Multi-modal 3D Semantic Segmentation in Real-world Autonomous Drivingcited by 2
Built on by
Citing papers per year: 2025: 1; 2026: 1.
2 papers.