mlsanityData to 7 Oct 2026

Bridged Transformer for Vision and Point Cloud 3D Object Detection

arXiv 2210.01391 ↗First posted 4 Oct 2022cs.CVcited by 19

Not on the map. Why?

Abstract (arXiv)

3D object detection is a crucial research topic in computer vision, which usually uses 3D point clouds as input in conventional setups. Recently, there is a trend of leveraging multiple sources of input data, such as complementing the 3D point cloud with 2D images that often have richer color and fewer noises. However, due to the heterogeneous geometrics of the 2D and 3D representations, it prevents us from applying off-the-shelf neural networks to achieve multimodal fusion. To that end, we propose Bridged Transformer (BrT), an end-to-end architecture for 3D object detection. BrT is simple and effective, which learns to identify 3D and 2D object bounding boxes from both points and image patches. A key element of BrT lies in the utilization of object queries for bridging 3D and 2D spaces, which unifies different sources of data representations in Transformer. We adopt a form of feature aggregation realized by point-to-patch projections which further strengthen the correlations between images and points. Moreover, BrT works seamlessly for fusing the point cloud with multi-view images. We experimentally show that BrT surpasses state-of-the-art methods on SUN RGB-D and ScanNetV2 datasets.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Builds on 33 of its 42 references that are in the map. First posted on arXiv 4 Oct 2022.

Builds on

33 of 42 references.

References · 33

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  2. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  3. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  4. 2020End-to-End Object Detection with Transformerscited by 7,384
  5. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  6. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  7. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  8. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,410
  9. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  10. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,923
  11. 2017Decoupled Weight Decay Regularizationcited by 1,722
  12. 2020Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformerscited by 1,368
21 more
  1. 2017VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detectioncited by 1,323
  2. 2017Multimodal Machine Learning: A Survey and Taxonomycited by 1,004
  3. 2016Multi-View 3D Object Detection Network for Autonomous Drivingcited by 721
  4. 2019Deep Hough Voting for 3D Object Detection in Point Cloudscited by 648
  5. 2017Frustum PointNets for 3D Object Detection from RGB-D Datacited by 640
  6. 2015Multi-view Convolutional Neural Networks for 3D Shape Recognitioncited by 564
  7. 2019Deep Learning for 3D Point Clouds: A Surveycited by 557
  8. 2021Conditional DETR for Fast Training Convergencecited by 422
  9. 2021An End-to-End Transformer Model for 3D Object Detectioncited by 329
  10. 2021Voxel Transformer for 3D Object Detectioncited by 247
  11. 2021Group-Free 3D Object Detection via Transformerscited by 215
  12. 2017PointFusion: Deep Sensor Fusion for 3D Bounding Box Estimationcited by 170
  13. 2021You Only Look at One Sequence: Rethinking Transformer in Vision through Object Detectioncited by 170
  14. 2021HRFormer: High-Resolution Transformer for Dense Predictioncited by 149
  15. 20203D-MPA: Multi Proposal Aggregation for 3D Semantic Instance Segmentationcited by 140
  16. 2020H3DNet: 3D Object Detection Using Hybrid Geometric Primitivescited by 123
  17. 2020ImVoteNet: Boosting 3D Object Detection in Point Clouds with Image Votescited by 119
  18. 2020Deep Multimodal Fusion by Channel Exchangingcited by 117
  19. 2015Deep Sliding Shapes for Amodal 3D Object Detection in RGB-D Imagescited by 111
  20. 2020MLCVNet: Multi-Level Context VoteNet for 3D Object Detectioncited by 80
  21. 2020Generative Sparse Detection Networks for 3D Single-shot Object Detectioncited by 69

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 10; 2024: 5; 2025: 1; 2026: 1.

19 papers.

Sort:

By area:

  1. 2023RangeViT: Towards Vision Transformers for 3D Semantic Segmentation in Autonomous Drivingcited by 76
  2. 2023C2Former: Calibrated and Complementary Transformer for RGB-Infrared Object Detectioncited by 57
  3. 20233D-STMN: Dependency-Driven Superpoint-Text Matching Network for End-to-End 3D Referring Expression Segmentationcited by 36
  4. 2023PanoContext-Former: Panoramic Total Scene Understanding with a Transformercited by 19
  5. 20232D-3D Interlaced Transformer for Point Cloud Segmentation with Scene-Level Supervisioncited by 16
  6. 2022A Comprehensive Survey of Transformers for Computer Visioncited by 13
  7. 20243DGS-DET: Empower 3D Gaussian Splatting with Boundary Guidance and Box-Focused Sampling for Indoor 3D Object Detectioncited by 13
  8. 2023Recent Advances in Multi-modal 3D Intelligence: A Comprehensive Survey and Evaluationcited by 12
  9. 2024FullLoRA: Efficiently Boosting the Robustness of Pretrained Vision Transformerscited by 9
  10. 2024Radar and Camera Fusion for Object Detection and Tracking: A Comprehensive Surveycited by 9
  11. 2023CoMAE: Single Model Hybrid Pre-training on Small-Scale RGB-D Datasetscited by 8
  12. 2023ViPFormer: Efficient Vision-and-Pointcloud Transformer for Unsupervised Pointcloud Understandingcited by 8
7 more
  1. 2024RGB2Point: 3D Point Cloud Generation from Single RGB Imagescited by 7
  2. 2023Efficient Transformer-based 3D Object Detection with Dynamic Token Haltingcited by 3
  3. 2022Position Embedding Needs an Independent Layer Normalizationcited by 1
  4. 2023SVT: Supertoken Video Transformer for Efficient Video Understandingcited by 1
  5. 2025High-quality Pseudo-labeling for Point Cloud Segmentation with Scene-level Annotationcited by 1
  6. 2026MedPCFM: Improving Medical Point Cloud Completion by Integrating Point Transformers and Flow Matchingcited by 1
  7. 2024Exploring The Neural Burden In Pruned Models: An Insight Inspired By Neurosciencecited by 0