mlsanityData to 7 Oct 2026

LWSIS: LiDAR-guided Weakly Supervised Instance Segmentation for Autonomous Driving

arXiv 2212.03504 ↗First posted 7 Dec 2022cs.CV cs.AIcited by 9

Not on the map. Why?

Abstract (arXiv)

Image instance segmentation is a fundamental research topic in autonomous driving, which is crucial for scene understanding and road safety. Advanced learning-based approaches often rely on the costly 2D mask annotations for training. In this paper, we present a more artful framework, LiDAR-guided Weakly Supervised Instance Segmentation (LWSIS), which leverages the off-the-shelf 3D data, i.e., Point Cloud, together with the 3D boxes, as natural weak supervisions for training the 2D image instance segmentation models. Our LWSIS not only exploits the complementary information in multimodal data during training, but also significantly reduces the annotation cost of the dense 2D masks. In detail, LWSIS consists of two crucial modules, Point Label Assignment (PLA) and Graph-based Consistency Regularization (GCR). The former module aims to automatically assign the 3D point cloud as 2D point-wise labels, while the latter further refines the predictions by enforcing geometry and appearance consistency of the multimodal data. Moreover, we conduct a secondary instance segmentation annotation on the nuScenes, named nuInsSeg, to encourage further research on multimodal perception tasks. Extensive experiments on the nuInsSeg, as well as the large-scale Waymo, show that LWSIS can substantially improve existing weakly supervised segmentation models by only involving 3D data during training. Additionally, LWSIS can also be incorporated into 3D object detectors like PointPainting to boost the 3D detection performance for free. The code and dataset are available at https://github.com/Serenos/LWSIS.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 28 of its 37 references that are in the map. First posted on arXiv 7 Dec 2022.

Builds on

28 of 37 references.

References · 28

  1. 2017Mask R-CNNcited by 7,259
  2. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  3. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,239
  4. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,436
  5. 2020Center-based 3D Object Detection and Trackingcited by 1,143
  6. 2019PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detectioncited by 890
  7. 2019Hybrid Task Cascade for Instance Segmentationcited by 479
  8. 2019PointPainting: Sequential Fusion for 3D Object Detectioncited by 449
  9. 2019PointRend: Image Segmentation as Renderingcited by 439
  10. 2020Conditional Convolutions for Instance Segmentationcited by 314
  11. 2015What's the Point: Semantic Segmentation with Point Supervisioncited by 297
  12. 2019Weakly Supervised Learning of Instance Segmentation with Inter-pixel Relationscited by 245
16 more
  1. 2016Simple Does It: Weakly Supervised Instance and Semantic Segmentationcited by 226
  2. 2020BoxInst: High-Performance Instance Segmentation with Box Annotationscited by 150
  3. 2020BlendMask: Top-Down Meets Bottom-Up for Instance Segmentationcited by 144
  4. 2018Normalized Cut Loss for Weakly-supervised CNN Segmentationcited by 112
  5. 2021BBAM: Bounding Box Attribution Map for Weakly Supervised Semantic and Instance Segmentationcited by 104
  6. 2019Box-driven Class-wise Region Masking and Filling Rate Guided Loss for Weakly Supervised Semantic Segmentationcited by 97
  7. 2021Leveraging Auxiliary Tasks with Affinity Learning for Weakly Supervised Semantic Segmentationcited by 77
  8. 2020Weakly Supervised 3D Object Detection from Lidar Point Cloudcited by 67
  9. 2021DiscoBox: Weakly Supervised Instance Segmentation and Semantic Correspondence from Box Supervisioncited by 59
  10. 2021RefineMask: Towards High-Quality Instance Segmentation with Fine-Grained Featurescited by 47
  11. 2022Semi-supervised 3D Object Detection with Proficient Teacherscited by 45
  12. 2022SSDA3D: Semi-supervised Domain Adaptation for 3D Object Detection from Point Cloudcited by 37
  13. 2020Weakly Supervised Instance Segmentation by Learning Annotation Consistent Instancescited by 31
  14. 2021Weakly-supervised Instance Segmentation via Class-agnostic Learning with Salient Imagescited by 20
  15. 2019Label-PEnet: Sequential Label Propagation and Enhancement Networks for Weakly Supervised Instance Segmentationcited by 17
  16. 2022Graph Neural Network and Spatiotemporal Transformer Attention for 3D Video Object Detection from Point Cloudscited by 17

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 2; 2025: 3; 2026: 1.

9 papers.

  1. 2024IS-Fusion: Instance-Scene Collaborative Fusion for Multimodal 3D Object Detectioncited by 57
  2. 2023Weakly Supervised Monocular 3D Object Detection using Multi-View Projection and Direction Consistencycited by 18
  3. 2023MWSIS: Multimodal Weakly Supervised Instance Segmentation with 2D Box Annotations for Autonomous Drivingcited by 5
  4. 2024RepVF: A Unified Vector Fields Representation for Multi-task 3D Perceptioncited by 5
  5. 2025SAM4D: Segment Anything in Camera and LiDAR Streamscited by 3
  6. 2023What Matters to Enhance Traffic Rule Compliance of Imitation Learning for End-to-End Autonomous Drivingcited by 2
  7. 2025When Small Guides Large: Cross-Model Co-Learning for Test-Time Adaptationcited by 0
  8. 2025LaGen: Towards Autoregressive LiDAR Scene Generationcited by 0
  9. 2026DeGuNet: Depth-Guided Ultra-Compact Backbones for Efficient LiDAR-Camera 3D Detectioncited by 0