mlsanityData to 14 Oct 2026

Depth- and Semantics-aware Multi-modal Domain Translation: Generating 3D Panoramic Color Images from LiDAR Point Clouds

arXiv 2302.07661 ↗First posted 15 Feb 2023cs.CVcited by 1

Not on the map. Why?

Abstract (arXiv)

This work presents a new depth- and semantics-aware conditional generative model, named TITAN-Next, for cross-domain image-to-image translation in a multi-modal setup between LiDAR and camera sensors. The proposed model leverages scene semantics as a mid-level representation and is able to translate raw LiDAR point clouds to RGB-D camera images by solely relying on semantic scene segments. We claim that this is the first framework of its kind and it has practical applications in autonomous vehicles such as providing a fail-safe mechanism and augmenting available data in the target image domain. The proposed model is evaluated on the large-scale and challenging Semantic-KITTI dataset, and experimental findings show that it considerably outperforms the original TITAN-Net and other strong baselines by 23.7% margin in terms of IoU.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 36 of its 52 references that are in the map. First posted on arXiv 15 Feb 2023.

Builds on

36 of 52 references.

References · 36

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  3. 2016Densely Connected Convolutional Networkscited by 6,720
  4. 2016Feature Pyramid Networks for Object Detectioncited by 5,036
  5. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,527
  6. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,444
  7. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,988
  8. 2017Improved Training of Wasserstein GANscited by 2,235
  9. 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,495
  10. 2019Semantic Image Synthesis with Spatially-Adaptive Normalizationcited by 1,299
  11. 2017High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANscited by 1,179
  12. 2017Unsupervised Learning of Depth and Ego-Motion from Videocited by 782
24 more
  1. 2018ADVENT: Adversarial Entropy Minimization for Domain Adaptation in Semantic Segmentationcited by 623
  2. 2020FDA: Fourier Domain Adaptation for Semantic Segmentationcited by 541
  3. 2019PointPainting: Sequential Fusion for 3D Object Detectioncited by 449
  4. 2018Video-to-Video Synthesiscited by 390
  5. 2020SalsaNext: Fast, Uncertainty-aware Semantic Segmentation of LiDAR Point Clouds for Autonomous Drivingcited by 226
  6. 2016Coupled Generative Adversarial Networkscited by 216
  7. 2020Hierarchical Multi-Scale Attention for Semantic Segmentationcited by 183
  8. 2017Learning Depth from Monocular Videos using Direct Methodscited by 154
  9. 2020EfficientPS: Efficient Panoptic Segmentationcited by 111
  10. 2019Learning to Predict Layout-to-image Conditional Convolutions for Semantic Image Synthesiscited by 108
  11. 2020World-Consistent Video-to-Video Synthesiscited by 69
  12. 2021Unsupervised Scale-consistent Depth Learning from Videocited by 69
  13. 2020SurfelGAN: Synthesizing Realistic Sensor Data for Autonomous Drivingcited by 68
  14. 2019SalsaNet: Fast Road and Vehicle Segmentation in LiDAR Point Clouds for Autonomous Drivingcited by 57
  15. 2019Densely Connected Search Space for More Flexible Neural Architecture Searchcited by 52
  16. 2021Virtual Normal: Enforcing Geometric Constraints for Accurate and Robust Depth Predictioncited by 48
  17. 2020SESAME: Semantic Editing of Scenes by Adding, Manipulating or Erasing Objectscited by 43
  18. 2018InstaGAN: Instance-aware Image-to-Image Translationcited by 37
  19. 2018Art2Real: Unfolding the Reality of Artworks via Semantically-Aware Image-to-Image Translationcited by 17
  20. 2019SDNet: Semantically Guided Depth Estimation Networkcited by 11
  21. 2021Semantics-aware Multi-modal Domain Translation:From LiDAR Point Clouds to Panoramic Color Imagescited by 6
  22. 2019Semantics-Aware Image to Image Translation and Domain Transfercited by 4
  23. 2020Integrating Categorical Semantics into Unsupervised Domain Translationcited by 4
  24. 2021Generating Photo-realistic Images from LiDAR Point Clouds with Generative Adversarial Networkscited by 2

Built on by

Cited by year

Citing papers per year: 2023: 1.

1 paper.

  1. 2023Semantics-aware LiDAR-Only Pseudo Point Cloud Generation for 3D Object Detectioncited by 0