mlsanityData to 7 Oct 2026

Semantics-aware Multi-modal Domain Translation:From LiDAR Point Clouds to Panoramic Color Images

arXiv 2106.13974 ↗First posted 26 Jun 2021cs.CVcited by 6

Not on the map. Why?

Abstract (arXiv)

In this work, we present a simple yet effective framework to address the domain translation problem between different sensor modalities with unique data formats. By relying only on the semantics of the scene, our modular generative framework can, for the first time, synthesize a panoramic color image from a given full 3D LiDAR point cloud. The framework starts with semantic segmentation of the point cloud, which is initially projected onto a spherical surface. The same semantic segmentation is applied to the corresponding camera image. Next, our new conditional generative model adversarially learns to translate the predicted LiDAR segment maps to the camera image counterparts. Finally, generated image segments are processed to render the panoramic scene images. We provide a thorough quantitative evaluation on the SemanticKitti dataset and show that our proposed framework outperforms other strong baseline models. Our source code is available at https://github.com/halmstad-University/TITAN-NET

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 22 of its 38 references that are in the map. First posted on arXiv 26 Jun 2021.

Builds on

22 of 38 references.

References · 22

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
  3. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
  4. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,967
  5. 2017Improved Training of Wasserstein GANscited by 2,223
  6. 2014Conditional Generative Adversarial Netscited by 2,152
  7. 2017High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANscited by 1,176
  8. 2017Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networkscited by 707
  9. 2018Video-to-Video Synthesiscited by 388
  10. 2015Deep multi-scale video prediction beyond mean square errorcited by 297
  11. 2020SalsaNext: Fast, Uncertainty-aware Semantic Segmentation of LiDAR Point Clouds for Autonomous Drivingcited by 226
  12. 2016Precomputed Real-Time Texture Synthesis with Markovian Generative Adversarial Networkscited by 192
10 more
  1. 2020Hierarchical Multi-Scale Attention for Semantic Segmentationcited by 183
  2. 2020World-Consistent Video-to-Video Synthesiscited by 69
  3. 2020SurfelGAN: Synthesizing Realistic Sensor Data for Autonomous Drivingcited by 68
  4. 2019SalsaNet: Fast Road and Vehicle Segmentation in LiDAR Point Clouds for Autonomous Drivingcited by 57
  5. 2018InstaGAN: Instance-aware Image-to-Image Translationcited by 37
  6. 2018Art2Real: Unfolding the Reality of Artworks via Semantically-Aware Image-to-Image Translationcited by 17
  7. 2019Stochastic Conditional Generative Networks with Basis Decompositioncited by 9
  8. 2019Semantics-Aware Image to Image Translation and Domain Transfercited by 4
  9. 2020Integrating Categorical Semantics into Unsupervised Domain Translationcited by 4
  10. 2018Diverse Conditional Image Generation by Stochastic Regression with Latent Drop-Out Codescited by 1

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 2; 2023: 3.

6 papers.

  1. 2021Transfer beyond the Field of View: Dense Panoramic Semantic Segmentation via Unsupervised Domain Adaptationcited by 34
  2. 2022Simple and Effective Synthesis of Indoor 3D Scenescited by 33
  3. 2023TriVol: Point Cloud Rendering via Triple Volumescited by 16
  4. 2022Deep Learning based Computer Vision Methods for Complex Traffic Environments Perception: A Reviewcited by 7
  5. 2023Depth- and Semantics-aware Multi-modal Domain Translation: Generating 3D Panoramic Color Images from LiDAR Point Cloudscited by 1
  6. 2023Semantics-aware LiDAR-Only Pseudo Point Cloud Generation for 3D Object Detectioncited by 0