mlsanityData to 7 Oct 2026

Bidirectional Projection Network for Cross Dimension Scene Understanding

arXiv 2103.14326 ↗First posted 26 Mar 2021cs.CVcited by 95

Not on the map. Why?

Abstract (arXiv)

2D image representations are in regular grids and can be processed efficiently, whereas 3D point clouds are unordered and scattered in 3D space. The information inside these two visual domains is well complementary, e.g., 2D images have fine-grained texture while 3D point clouds contain plentiful geometry information. However, most current visual recognition systems process them individually. In this paper, we present a bidirectional projection network (BPNet) for joint 2D and 3D reasoning in an end-to-end manner. It contains 2D and 3D sub-networks with symmetric architectures, that are connected by our proposed bidirectional projection module (BPM). Via the BPM, complementary 2D and 3D information can interact with each other in multiple architectural levels, such that advantages in these two visual domains can be combined for better scene recognition. Extensive quantitative and qualitative experimental evaluations show that joint reasoning over 2D and 3D visual domains can benefit both 2D and 3D scene understanding simultaneously. Our BPNet achieves top performance on the ScanNetV2 benchmark for both 2D and 3D semantic segmentation. Code is available at urlhttps://github.com/wbhu/BPNet.

Key facts. Cited by 95 papers in the map (data through 7 Oct 2026). Builds on 57 of its 72 references that are in the map. First posted on arXiv 26 Mar 2021.

Builds on

57 of 72 references.

References · 57

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  4. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  5. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  6. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,845
  7. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  8. 2014Going Deeper with Convolutionscited by 4,231
  9. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,239
  10. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  11. 2016Pyramid Scene Parsing Networkcited by 2,472
  12. 2018Dynamic Graph CNN for Learning on Point Cloudscited by 2,220
38 more
  1. 2015SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentationcited by 1,483
  2. 2019KPConv: Flexible and Deformable Convolution for Point Cloudscited by 1,202
  3. 20194D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networkscited by 1,201
  4. 2015Multi-Scale Context Aggregation by Dilated Convolutionscited by 1,128
  5. 2018PointCNN: Convolution On X-Transformed Pointscited by 764
  6. 20173D Semantic Segmentation with Submanifold Sparse Convolutional Networkscited by 746
  7. 2018CCNet: Criss-Cross Attention for Semantic Segmentationcited by 717
  8. 2018PointConv: Deep Convolutional Networks on 3D Point Cloudscited by 656
  9. 2019RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Cloudscited by 584
  10. 2015Multi-view Convolutional Neural Networks for 3D Shape Recognitioncited by 564
  11. 2014Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFscited by 552
  12. 2019DeepGCNs: Can GCNs Go as Deep as CNNs?cited by 527
  13. 2016Semantic Scene Completion from a Single Depth Imagecited by 468
  14. 2015Learning Deconvolution Network for Semantic Segmentationcited by 365
  15. 2016OctNet: Learning Deep 3D Representations at High Resolutionscited by 340
  16. 2018Context Encoding for Semantic Segmentationcited by 289
  17. 2017ICNet for Real-Time Semantic Segmentation on High-Resolution Imagescited by 261
  18. 2018SpiderCNN: Deep Learning on Point Sets with Parameterized Convolutional Filterscited by 261
  19. 2016Volumetric and Multi-View CNNs for Object Classification on 3D Datacited by 240
  20. 2020PointASNL: Robust Point Clouds Processing using Nonlocal Neural Networks with Adaptive Samplingcited by 238
  21. 2017SEGCloud: Semantic Segmentation of 3D Point Cloudscited by 178
  22. 2020OccuSeg: Occupancy-aware 3D Instance Segmentationcited by 177
  23. 2015ParseNet: Looking Wider to See Bettercited by 176
  24. 2021Deep Parametric Continuous Convolutional Neural Networkscited by 161
  25. 2018SPLATNet: Sparse Lattice Networks for Point Cloud Processingcited by 158
  26. 2016SemanticFusion: Dense 3D Semantic Mapping with Convolutional Neural Networkscited by 134
  27. 2019xMUDA: Cross-Modal Unsupervised Domain Adaptation for 3D Semantic Segmentationcited by 129
  28. 2017SurfaceNet: An End-to-end 3D Neural Network for Multiview Stereopsiscited by 128
  29. 2014Learning Rich Features from RGB-D Images for Object Detection and Segmentationcited by 124
  30. 20183DMV: Joint 3D-Multi-View Prediction for 3D Semantic Scene Segmentationcited by 118
  31. 2020Virtual Multi-view Fusion for 3D Semantic Segmentationcited by 99
  32. 2018Local Spectral Graph Convolution for Point Set Feature Learningcited by 84
  33. 2019Multi-view PointNet for 3D Scene Understandingcited by 76
  34. 2018Depth-aware CNN for RGB-D Segmentationcited by 73
  35. 2018Self-Supervised Model Adaptation for Multimodal Semantic Segmentationcited by 73
  36. 2019Hierarchical Point-Edge Interaction Network for Point Cloud Semantic Segmentationcited by 68
  37. 2018Monte Carlo Convolution for Learning on Non-Uniformly Sampled Point Cloudscited by 62
  38. 2020JSENet: Joint Semantic Segmentation and Edge Detection Network for 3D Point Cloudscited by 62

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 12; 2022: 22; 2023: 19; 2024: 22; 2025: 15; 2026: 5.

95 papers.

Sort:

By area:

  1. 2022OpenScene: 3D Scene Understanding with Open Vocabulariescited by 556
  2. 2022CMX: Cross-Modal Fusion for RGB-X Semantic Segmentation with Transformerscited by 229
  3. 2024DepthCrafter: Generating Consistent Long Depth Sequences for Open-world Videoscited by 202
  4. 2021Mix3D: Out-of-Context Data Augmentation for 3D Scenescited by 126
  5. 2022Neural 3D Scene Reconstruction with the Manhattan-world Assumptioncited by 123
  6. 2021Fast Point Transformercited by 103
  7. 2024SAMPart3D: Segment Any Part in 3D Objectscited by 95
  8. 2022X-Trans2Cap: Cross-Modal Knowledge Transfer using Transformer for 3D Dense Captioningcited by 70
  9. 2021Pri3D: Can 3D Priors Help 2D Representation Learning?cited by 61
  10. 2024MaskClustering: View Consensus based Mask Graph Clustering for Open-Vocabulary 3D Instance Segmentationcited by 56
  11. 20214DContrast: Contrastive Learning with Dynamic Correspondences for 3D Scene Understandingcited by 54
  12. 2022Learning Multi-View Aggregation In the Wild for Large-Scale 3D Semantic Segmentationcited by 47
38 more
  1. 2025DINO in the Room: Leveraging 2D Foundation Models for 3D Segmentationcited by 31
  2. 2024PO3AD: Predicting Point Offsets toward Better 3D Point Cloud Anomaly Detectioncited by 30
  3. 2022PointConvFormer: Revenge of the Point-based Convolutioncited by 29
  4. 2022Divide and Conquer: 3D Point Cloud Instance Segmentation With Point-Wise Binarizationcited by 26
  5. 2024ODIN: A Single Model for 2D and 3D Segmentationcited by 25
  6. 2021Voint Cloud: Multi-View Point Cloud Representation for 3D Understandingcited by 22
  7. 2024Open-World Semantic Segmentation Including Class Similaritycited by 22
  8. 2022VIBUS: Data-efficient 3D Scene Parsing with VIewpoint Bottleneck and Uncertainty-Spectrum Modelingcited by 21
  9. 2023Hierarchical Point-based Active Learning for Semi-supervised Point Cloud Semantic Segmentationcited by 21
  10. 2022Point Scene Understanding via Disentangled Instance Mesh Reconstructioncited by 20
  11. 20233DCoMPaT++: An improved Large-scale 3D Vision Dataset for Compositional Recognitioncited by 20
  12. 2024Region-Based Representations Revisitedcited by 18
  13. 2024Open Vocabulary 3D Scene Understanding via Geometry Guided Self-Distillationcited by 17
  14. 20232D-3D Interlaced Transformer for Point Cloud Segmentation with Scene-Level Supervisioncited by 16
  15. 2024KPConvX: Modernizing Kernel Point Convolution with Kernel Attentioncited by 16
  16. 2024Dense Multimodal Alignment for Open-Vocabulary 3D Scene Understandingcited by 15
  17. 2023PointMBF: A Multi-scale Bidirectional Fusion Network for Unsupervised RGB-D Point Cloud Registrationcited by 14
  18. 2024FMM-Attack: A Flow-based Multi-modal Adversarial Attack on Video-based LLMscited by 14
  19. 2023PCR-CG: Point Cloud Registration via Deep Explicit Color and Geometrycited by 13
  20. 2025Cross-Modal and Uncertainty-Aware Agglomeration for Open-Vocabulary 3D Scene Understandingcited by 13
  21. 2026Any3D-VLA: Enhancing VLA Robustness via Diverse Point Cloudscited by 13
  22. 2021A Real-Time Online Learning Framework for Joint 3D Reconstruction and Semantic Segmentation of Indoor Scenescited by 12
  23. 2023Recent Advances in Multi-modal 3D Intelligence: A Comprehensive Survey and Evaluationcited by 12
  24. 20243D Weakly Supervised Semantic Segmentation with 2D Vision-Language Guidancecited by 12
  25. 2022SemAffiNet: Semantic-Affine Transformation for Point Cloud Segmentationcited by 11
  26. 2024Grounding 3D Scene Affordance From Egocentric Interactionscited by 11
  27. 2022TO-Scene: A Large-scale Dataset for Understanding 3D Tabletop Scenescited by 10
  28. 2023Mx2M: Masked Cross-Modality Modeling in Domain Adaptation for 3D Semantic Segmentationcited by 10
  29. 2021HIDA: Towards Holistic Indoor Understanding for the Visually Impaired via Semantic Instance Segmentation with a Wearable Solid-State LiDAR Sensorcited by 9
  30. 2022GaIA: Graphical Information Gain based Attention Network for Weakly Supervised Point Cloud Semantic Segmentationcited by 9
  31. 2023Clutter Detection and Removal in 3D Scenes with View-Consistent Inpaintingcited by 9
  32. 2023Joint Depth Prediction and Semantic Segmentation with Multi-View SAMcited by 8
  33. 2022Self-Supervised Image Representation Learning with Geometric Set Consistencycited by 7
  34. 2023SSR-2D: Semantic 3D Scene Reconstruction from 2D Imagescited by 7
  35. 2024Video Watermarking: Safeguarding Your Video from (Unauthorized) Annotations by Video-based LLMscited by 7
  36. 2023Region-Enhanced Feature Learning for Scene Semantic Segmentationcited by 6
  37. 2023Retro-FPN: Retrospective Feature Pyramid Network for Point Cloud Semantic Segmentationcited by 6
  38. 2023ALSTER: A Local Spatio-Temporal Expert for Online 3D Semantic Reconstructioncited by 6

50 most cited of 95 All 95 (JSON)