mlsanityData to 7 Oct 2026

Virtual Multi-view Fusion for 3D Semantic Segmentation

arXiv 2007.13138 ↗First posted 26 Jul 2020cs.CV eess.IVcited by 99

Not on the map. Why?

Abstract (arXiv)

Semantic segmentation of 3D meshes is an important problem for 3D scene understanding. In this paper we revisit the classic multiview representation of 3D meshes and study several techniques that make them effective for 3D semantic segmentation of meshes. Given a 3D mesh reconstructed from RGBD sensors, our method effectively chooses different virtual views of the 3D mesh and renders multiple 2D channels for training an effective 2D semantic segmentation model. Features from multiple per view predictions are finally fused on 3D mesh vertices to predict mesh semantic segmentation labels. Using the large scale indoor 3D semantic segmentation benchmark of ScanNet, we show that our virtual views enable more effective training of 2D semantic segmentation networks than previous multiview approaches. When the 2D per pixel predictions are aggregated on 3D surfaces, our virtual multiview fusion method is able to achieve significantly better 3D semantic segmentation results compared to all prior multiview approaches and competitive with recent 3D convolution approaches.

Key facts. Cited by 99 papers in the map (data through 7 Oct 2026). First posted on arXiv 26 Jul 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 23; 2022: 20; 2023: 24; 2024: 14; 2025: 13; 2026: 5.

99 papers.

Sort:

By area:

  1. 2022OpenScene: 3D Scene Understanding with Open Vocabulariescited by 556
  2. 2023SAM3D: Segment Anything in 3D Scenescited by 155
  3. 2021Mix3D: Out-of-Context Data Augmentation for 3D Scenescited by 126
  4. 2022Neural 3D Scene Reconstruction with the Manhattan-world Assumptioncited by 123
  5. 2021NeSF: Neural Semantic Fields for Generalizable Semantic Segmentation of 3D Scenescited by 119
  6. 2021LENS: Localization enhanced by NeRF synthesiscited by 116
  7. 2021Fast Point Transformercited by 103
  8. 2021Bidirectional Projection Network for Cross Dimension Scene Understandingcited by 95
  9. 2023SAI3D: Segment Any Instance in 3D Scenescited by 90
  10. 2021One Thing One Click: A Self-Training Approach for Weakly Supervised 3D Semantic Segmentationcited by 89
  11. 2021Guided Point Contrastive Learning for Semi-supervised Point Cloud Semantic Segmentationcited by 87
  12. 2023OpenIns3D: Snap and Lookup for 3D Open-vocabulary Instance Segmentationcited by 82
38 more
  1. 2022SurroundDepth: Entangling Surrounding Views for Self-Supervised Multi-Camera Depth Estimationcited by 76
  2. 2021SQN: Weakly-Supervised Semantic Segmentation of Large-Scale 3D Point Cloudscited by 64
  3. 2021Learning 3D Semantic Segmentation with only 2D Image Supervisioncited by 60
  4. 2022Contrastive Boundary Learning for Point Cloud Segmentationcited by 56
  5. 20214DContrast: Contrastive Learning with Dynamic Correspondences for 3D Scene Understandingcited by 54
  6. 20213D-to-2D Distillation for Indoor Scene Parsingcited by 47
  7. 2022Learning Multi-View Aggregation In the Wild for Large-Scale 3D Semantic Segmentationcited by 47
  8. 2023Nerflets: Local Radiance Fields for Efficient Structure-Aware 3D Scene Representation from 2D Supervisioncited by 44
  9. 2023SAMPro3D: Locating SAM Prompts in 3D for Zero-Shot Instance Segmentationcited by 44
  10. 2021SAIL-VOS 3D: A Synthetic Dataset and Baselines for Object Detection and 3D Mesh Reconstruction from Video Datacited by 38
  11. 2021RandomRooms: Unsupervised Pre-training from Synthetic Shapes and Randomized Layouts for 3D Object Detectioncited by 37
  12. 2023GrowSP: Unsupervised Semantic Segmentation of 3D Point Cloudscited by 34
  13. 2021BuildingNet: Learning to Label 3D Buildingscited by 33
  14. 2021Learning from 2D: Contrastive Pixel-to-Point Knowledge Transfer for 3D Pretrainingcited by 32
  15. 2025DINO in the Room: Leveraging 2D Foundation Models for 3D Segmentationcited by 31
  16. 2022PointConvFormer: Revenge of the Point-based Convolutioncited by 29
  17. 2023S4C: Self-Supervised Semantic Scene Completion with Neural Fieldscited by 27
  18. 2022Divide and Conquer: 3D Point Cloud Instance Segmentation With Point-Wise Binarizationcited by 26
  19. 2024ODIN: A Single Model for 2D and 3D Segmentationcited by 25
  20. 2021Voint Cloud: Multi-View Point Cloud Representation for 3D Understandingcited by 22
  21. 2024Open-World Semantic Segmentation Including Class Similaritycited by 22
  22. 2023Hierarchical Point-based Active Learning for Semi-supervised Point Cloud Semantic Segmentationcited by 21
  23. 2024Multimodality Helps Few-shot 3D Point Cloud Semantic Segmentationcited by 20
  24. 2024Region-Based Representations Revisitedcited by 18
  25. 20232D-3D Interlaced Transformer for Point Cloud Segmentation with Scene-Level Supervisioncited by 16
  26. 2022Dual Adaptive Transformations for Weakly Supervised Point Cloud Segmentationcited by 15
  27. 2025Unifying 2D and 3D Vision-Language Understandingcited by 15
  28. 2024iSeg: Interactive 3D Segmentation via Interactive Attentioncited by 14
  29. 2023All Points Matter: Entropy-Regularized Distribution Alignment for Weakly-supervised 3D Segmentationcited by 13
  30. 2023Neural Semantic Surface Mapscited by 13
  31. 2021A Real-Time Online Learning Framework for Joint 3D Reconstruction and Semantic Segmentation of Indoor Scenescited by 12
  32. 2022Box2Seg: Learning Semantics of 3D Point Clouds with Box-Level Supervisioncited by 11
  33. 2022Beyond RGB: Scene-Property Synthesis with Neural Radiance Fieldscited by 10
  34. 2022MvDeCor: Multi-view Dense Correspondence Learning for Fine-grained 3D Segmentationcited by 10
  35. 2021Exploiting Local Geometry for Feature and Graph Construction for Better 3D Point Cloud Processing with Graph Neural Networkscited by 9
  36. 2023Clutter Detection and Removal in 3D Scenes with View-Consistent Inpaintingcited by 9
  37. 2024PointSeg: A Training-Free Paradigm for 3D Scene Segmentation via Foundation Modelscited by 9
  38. 2024PARIS3D: Reasoning-based 3D Part Segmentation Using Large Multimodal Modelcited by 9

50 most cited of 99 All 99 (JSON)