mlsanityData to 7 Oct 2026

VioLA: Aligning Videos to 2D LiDAR Scans

arXiv 2311.04783 ↗First posted 8 Nov 2023cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

We study the problem of aligning a video that captures a local portion of an environment to the 2D LiDAR scan of the entire environment. We introduce a method (VioLA) that starts with building a semantic map of the local scene from the image sequence, then extracts points at a fixed height for registering to the LiDAR map. Due to reconstruction errors or partial coverage of the camera scan, the reconstructed semantic map may not contain sufficient information for registration. To address this problem, VioLA makes use of a pre-trained text-to-image inpainting model paired with a depth completion model for filling in the missing scene content in a geometrically consistent fashion to support pose registration. We evaluate VioLA on two real-world RGB-D benchmarks, as well as a self-captured dataset of a large office scene. Notably, our proposed scene completion module improves the pose registration performance by up to 20%.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 42 of its 64 references that are in the map. First posted on arXiv 8 Nov 2023.

Builds on

42 of 64 references.

References · 42

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  2. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  3. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  4. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,338
  5. 2021DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Camerascited by 673
  6. 2016Semantic Scene Completion from a Single Depth Imagecited by 468
  7. 2021MonoScene: Monocular 3D Semantic Scene Completioncited by 357
  8. 20163DMatch: Learning Local Geometric Descriptors from RGB-D Reconstructionscited by 333
  9. 2019Deep Closest Point: Learning Representations for Point Cloud Registrationcited by 318
  10. 2020PREDATOR: Registration of 3D Point Clouds with Low Overlapcited by 287
  11. 2021PoinTr: Diverse Point Cloud Completion with Geometry-Aware Transformerscited by 284
  12. 2020TEASER: Fast and Certifiable Point Cloud Registrationcited by 274
30 more
  1. 2019PointNetLK: Robust & Efficient Point Cloud Registration using PointNetcited by 231
  2. 2020Deep Global Registrationcited by 228
  3. 2022AutoSDF: Shape Priors for 3D Completion, Reconstruction and Generationcited by 208
  4. 2022Geometric Transformer for Fast and Robust Point Cloud Registrationcited by 200
  5. 2020RPM-Net: Robust Point Matching using Learned Featurescited by 181
  6. 2018PPFNet: Global Context Aware Local Features for Robust 3D Point Matchingcited by 180
  7. 2022Panoptic Lifting for 3D Scene Understanding with Neural Fieldscited by 171
  8. 2018The Perfect Match: 3D Point Cloud Matching with Smoothed Densitiescited by 169
  9. 2023SceneScape: Text-Driven Consistent Scene Generationcited by 160
  10. 2019PRNet: Self-Supervised Learning for Partial-to-Partial Registrationcited by 146
  11. 2022ShapeFormer: Transformer-based Shape Completion via Sparse Representationcited by 106
  12. 2021Robust Point Cloud Registration Framework Based on Deep Graph Matchingcited by 96
  13. 2022Semantic Abstraction: Open-World 3D Scene Understanding from 2D Vision-Language Modelscited by 95
  14. 2023SCPNet: Semantic Scene Completion on Point Cloudcited by 80
  15. 2021DeepI2P: Image-to-Point Cloud Registration via Deep Classificationcited by 58
  16. 2021Semantic Scene Completion via Integrating Instances and Scene in-the-Loopcited by 54
  17. 2021View-Guided Point Cloud Completioncited by 46
  18. 2022CorrI2P: Deep Image-to-Point Cloud Registration via Dense Correspondencecited by 43
  19. 2019A Polynomial-time Solution for Robust Registration with Extreme Outlier Ratescited by 38
  20. 2022IronDepth: Iterative Refinement of Single-View Depth using Surface Normal and its Uncertaintycited by 30
  21. 2021Sampling Network Guided Cross-Entropy Method for Unsupervised Point Cloud Registrationcited by 24
  22. 2020Monocular Camera Localization in Prior LiDAR Maps with 2D-3D Line Correspondencescited by 23
  23. 2023Pointersect: Neural Rendering with Cloud-Ray Intersectioncited by 23
  24. 2023Semantic Scene Completion with Cleaner Selfcited by 12
  25. 2016Exploiting 2D Floorplan for Building-scale Panorama RGBD Alignmentcited by 7
  26. 2022Category-Level Global Camera Pose Estimation with Multi-Hypothesis Point Cloud Correspondencescited by 3
  27. 2022Deep Confidence Guided Distance for 3D Partial Shape Registrationcited by 2
  28. 2022Floorplan-Aware Camera Poses Refinementcited by 2
  29. 20233D Surface Reconstruction in the Wild by Deforming Shape Priors from Synthetic Datacited by 2
  30. 2023Poses as Queries: Image-to-LiDAR Map Localization with Transformerscited by 2

Built on by

Not cited yet.