mlsanityData to 14 Oct 2026

Towards Learning to Complete Anything in Lidar

arXiv 2504.12264 ↗First posted 16 Apr 2025cs.CVcited by 7

Not on the map. Why?

Abstract (arXiv)

We propose CAL (Complete Anything in Lidar) for Lidar-based shape-completion in-the-wild. This is closely related to Lidar-based semantic/panoptic scene completion. However, contemporary methods can only complete and recognize objects from a closed vocabulary labeled in existing Lidar datasets. Different to that, our zero-shot approach leverages the temporal context from multi-modal sensor sequences to mine object shapes and semantic features of observed objects. These are then distilled into a Lidar-only instance-level completion and recognition model. Although we only mine partial shape completions, we find that our distilled model learns to infer full object shapes from multiple such partial observations across the dataset. We show that our model can be prompted on standard benchmarks for Semantic and Panoptic Scene Completion, localize objects as (amodal) 3D bounding boxes, and recognize objects beyond fixed class vocabularies. Our project page is https://research.nvidia.com/labs/dvl/projects/complete-anything-lidar

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 58 of its 74 references that are in the map. First posted on arXiv 16 Apr 2025.

Builds on

58 of 74 references.

References · 58

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2023Segment Anythingcited by 8,234
  3. 2017Neural Discrete Representation Learningcited by 4,712
  4. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,117
  5. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,897
  6. 2024SAM 2: Segment Anything in Images and Videoscited by 2,863
  7. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,689
  8. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,351
  9. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,041
  10. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,540
  11. 2017VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detectioncited by 1,323
  12. 20194D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networkscited by 1,208
38 more
  1. 2020Center-based 3D Object Detection and Trackingcited by 1,144
  2. 2018Panoptic Segmentationcited by 802
  3. 2021Extract Free Dense Labels from CLIPcited by 583
  4. 2021KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3Dcited by 575
  5. 2022OpenScene: 3D Scene Understanding with Open Vocabulariescited by 559
  6. 2016Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modelingcited by 558
  7. 2022LION: Latent Point Diffusion Models for 3D Shape Generationcited by 553
  8. 2016Semantic Scene Completion from a Single Depth Imagecited by 472
  9. 2020Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR Segmentationcited by 389
  10. 2018PCN: Point Completion Networkcited by 384
  11. 2023Occ3D: A Large-Scale 3D Occupancy Prediction Benchmark for Autonomous Drivingcited by 332
  12. 2023OpenMask3D: Open-Vocabulary 3D Instance Segmentationcited by 289
  13. 2020Sparse Single Sweep LiDAR Point Cloud Segmentation via Learning Contextual Shape Priors from Scene Completioncited by 223
  14. 2016Shape Completion using 3D-Encoder-Predictor CNNs and Shape Synthesiscited by 219
  15. 2021Group-Free 3D Object Detection via Transformerscited by 215
  16. 2023XCube: Large-Scale 3D Generative Modeling using Sparse Voxel Hierarchiescited by 189
  17. 2020LMSCNet: Lightweight Multiscale 3D Semantic Completioncited by 188
  18. 2021Panoptic nuScenes: A Large-Scale Benchmark for LiDAR Panoptic Segmentation and Trackingcited by 166
  19. 2022Open-Vocabulary Universal Image Segmentation with MaskCLIPcited by 132
  20. 2023Segment Any Point Cloud Sequences by Distilling Vision Foundation Modelscited by 125
  21. 2017ScanComplete: Large-Scale Scene Completion and Semantic Segmentation for 3D Scanscited by 103
  22. 2023SCPNet: Semantic Scene Completion on Point Cloudcited by 80
  23. 2019SG-NN: Sparse Generative Neural Networks for Self-Supervised Scene Completion of RGB-D Scanscited by 77
  24. 2023SSCBench: A Large-Scale 3D Semantic Scene Completion Benchmark for Autonomous Drivingcited by 75
  25. 2020Semantic Scene Completion using Local Deep Implicit Functions on LiDAR Datacited by 74
  26. 2022Scribble-Supervised LiDAR Semantic Segmentationcited by 69
  27. 20223D-LDM: Neural Implicit 3D Shape Generation with Latent Diffusion Modelscited by 67
  28. 2022Motion Inspired Unsupervised Perception and Prediction in Autonomous Drivingcited by 59
  29. 2024POP-3D: Open-Vocabulary 3D Occupancy Prediction from Imagescited by 52
  30. 2023AutoDecoding Latent 3D Diffusion Modelscited by 50
  31. 2018Two Stream 3D Semantic Scene Completioncited by 48
  32. 2022Dynamic 3D Scene Analysis by Point Cloud Accumulationcited by 47
  33. 2024Better Call SAL: Towards Learning to Segment Anything in Lidarcited by 47
  34. 2023PaSCo: Urban 3D Panoptic Scene Completion with Uncertainty Awarenesscited by 42
  35. 2020A Benchmark for LiDAR-based Panoptic Segmentation based on KITTIcited by 40
  36. 2022Towards Long-Tailed 3D Detectioncited by 40
  37. 2024Scaling Diffusion Models to Real-World 3D LiDAR Scene Completioncited by 40
  38. 2023Unsupervised 3D Perception with 2D Vision-Language Distillation for Autonomous Drivingcited by 37

50 most cited of 58 All 58 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 5; 2026: 2.

7 papers.

  1. 2025RF-DETR: Neural Architecture Search for Real-Time Detection Transformerscited by 75
  2. 2025Roboflow100-VL: A Multi-Domain Object Detection Benchmark for Vision-Language Modelscited by 20
  3. 2025RefAV: Towards Planning-Centric Scenario Miningcited by 13
  4. 2025IPFormer: Visual 3D Panoptic Scene Completion with Context-Adaptive Instance Proposalscited by 3
  5. 2025ShelfGaussian: Shelf-Supervised Open-Vocabulary Gaussian-based 3D Scene Understandingcited by 3
  6. 2026DetPO: In-Context Learning with Multi-Modal LLMs for Few-Shot Object Detectioncited by 2
  7. 2026OV3D-Bench: A Diagnostic Benchmark for Open-Vocabulary Monocular 3D Detectioncited by 0