# Back-tracing Representative Points for Voting-based 3D Object Detection in Point Clouds

arXiv: https://arxiv.org/abs/2104.06114 · HTML: https://mlsanity.com/paper/2104.06114/

## Key facts

- Cited by 67 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 44 references that are in the map.
- First posted on arXiv 13 Apr 2021.

## Builds on (most cited first, 29 of 29)

- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [PointPillars: Fast Encoders for Object Detection from Point Clouds](https://mlsanity.com/paper/1812.05784/) (2018)
- [FCOS: Fully Convolutional One-Stage Object Detection](https://mlsanity.com/paper/1904.01355/) (2019)
- [YOLO9000: Better, Faster, Stronger](https://mlsanity.com/paper/1612.08242/) (2016)
- [VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection](https://mlsanity.com/paper/1711.06396/) (2017)
- [PointRCNN: 3D Object Proposal Generation and Detection from Point Cloud](https://mlsanity.com/paper/1812.04244/) (2018)
- [PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detection](https://mlsanity.com/paper/1912.13192/) (2019)
- [CornerNet: Detecting Objects as Paired Keypoints](https://mlsanity.com/paper/1808.01244/) (2018)
- [Multi-View 3D Object Detection Network for Autonomous Driving](https://mlsanity.com/paper/1611.07759/) (2016)
- [CenterNet: Keypoint Triplets for Object Detection](https://mlsanity.com/paper/1904.08189/) (2019)
- [Deep Hough Voting for 3D Object Detection in Point Clouds](https://mlsanity.com/paper/1904.09664/) (2019)
- [Frustum PointNets for 3D Object Detection from RGB-D Data](https://mlsanity.com/paper/1711.08488/) (2017)
- [3DSSD: Point-based 3D Single Stage Object Detector](https://mlsanity.com/paper/2002.10187/) (2020)
- [From Points to Parts: 3D Object Detection from Point Cloud with Part-aware and Part-aggregation Network](https://mlsanity.com/paper/1907.03670/) (2019)
- [RepPoints: Point Set Representation for Object Detection](https://mlsanity.com/paper/1904.11490/) (2019)
- [3D-SIS: 3D Semantic Instance Segmentation of RGB-D Scans](https://mlsanity.com/paper/1812.07003/) (2018)
- [Bottom-up Object Detection by Grouping Extreme and Center Points](https://mlsanity.com/paper/1901.08043/) (2019)
- [GSPN: Generative Shape Proposal Network for 3D Instance Segmentation in Point Cloud](https://mlsanity.com/paper/1812.03320/) (2018)
- [3D-MPA: Multi Proposal Aggregation for 3D Semantic Instance Segmentation](https://mlsanity.com/paper/2003.13867/) (2020)
- [H3DNet: 3D Object Detection Using Hybrid Geometric Primitives](https://mlsanity.com/paper/2006.05682/) (2020)
- [Deep Sliding Shapes for Amodal 3D Object Detection in RGB-D Images](https://mlsanity.com/paper/1511.02300/) (2015)
- [MLCVNet: Multi-Level Context VoteNet for 3D Object Detection](https://mlsanity.com/paper/2004.05679/) (2020)
- [DeNet: Scalable Real-time Object Detection with Directed Sparse Sampling](https://mlsanity.com/paper/1703.10295/) (2017)

## Built on by (most cited first, 30 of 67)

- [Delving into the Devils of Bird's-eye-view Perception: A Review, Evaluation and Recipe](https://mlsanity.com/paper/2209.05324/) (2022)
- [DSVT: Dynamic Sparse Voxel Transformer with Rotated Sets](https://mlsanity.com/paper/2301.06051/) (2023)
- [3D-SPS: Single-Stage 3D Visual Grounding via Referred Point Progressive Selection](https://mlsanity.com/paper/2204.06272/) (2022)
- [OctFormer: Octree-based Transformers for 3D Point Clouds](https://mlsanity.com/paper/2305.03045/) (2023)
- [FCAF3D: Fully Convolutional Anchor-Free 3D Object Detection](https://mlsanity.com/paper/2112.00322/) (2021)
- [Surface Representation for Point Clouds](https://mlsanity.com/paper/2205.05740/) (2022)
- [UniTR: A Unified and Efficient Multi-Modal Transformer for Bird's-Eye-View Representation](https://mlsanity.com/paper/2308.07732/) (2023)
- [UniT3D: A Unified Transformer for 3D Dense Captioning and Visual Grounding](https://mlsanity.com/paper/2212.00836/) (2022)
- [CAGroup3D: Class-Aware Grouping for 3D Object Detection on Point Clouds](https://mlsanity.com/paper/2210.04264/) (2022)
- [CoDA: Collaborative Novel Box Discovery and Cross-modal Alignment for Open-vocabulary 3D Object Detection](https://mlsanity.com/paper/2310.02960/) (2023)
- [Implicit Autoencoder for Point-Cloud Self-Supervised Representation Learning](https://mlsanity.com/paper/2201.00785/) (2022)
- [VL-SAT: Visual-Linguistic Semantics Assisted Training for 3D Semantic Scene Graph Prediction in Point Cloud](https://mlsanity.com/paper/2303.14408/) (2023)
- [Graph R-CNN: Towards Accurate 3D Object Detection with Semantic-Decorated Local Graph](https://mlsanity.com/paper/2208.03624/) (2022)
- [RBGNet: Ray-based Grouping for 3D Object Detection](https://mlsanity.com/paper/2204.02251/) (2022)
- [LION: Linear Group RNN for 3D Object Detection in Point Clouds](https://mlsanity.com/paper/2407.18232/) (2024)
- [V-DETR: DETR with Vertex Relative Position Encoding for 3D Object Detection](https://mlsanity.com/paper/2308.04409/) (2023)
- [TR3D: Towards Real-Time Indoor 3D Object Detection](https://mlsanity.com/paper/2302.02858/) (2023)
- [Uni3DETR: Unified 3D Detection Transformer](https://mlsanity.com/paper/2310.05699/) (2023)
- [UniDet3D: Multi-dataset Indoor 3D Object Detection](https://mlsanity.com/paper/2409.04234/) (2024)
- [VoteHMR: Occlusion-Aware Voting Network for Robust 3D Human Mesh Recovery from Partial Point Clouds](https://mlsanity.com/paper/2110.08729/) (2021)
- [3D-GRES: Generalized 3D Referring Expression Segmentation](https://mlsanity.com/paper/2407.20664/) (2024)
- [Collaborative Novel Object Discovery and Box-Guided Cross-Modal Alignment for Open-Vocabulary 3D Object Detection](https://mlsanity.com/paper/2406.00830/) (2024)
- [ConDaFormer: Disassembled Transformer with Local Structure Enhancement for 3D Point Cloud Understanding](https://mlsanity.com/paper/2312.11112/) (2023)
- [HyperDet3D: Learning a Scene-conditioned 3D Object Detector](https://mlsanity.com/paper/2204.05599/) (2022)
- [ImGeoNet: Image-induced Geometry-aware Voxel Representation for Multi-view 3D Object Detection](https://mlsanity.com/paper/2308.09098/) (2023)
- [ARM3D: Attention-based relation module for indoor 3D object detection](https://mlsanity.com/paper/2202.09715/) (2022)
- [Boosting 3D Object Detection via Object-Focused Image Fusion](https://mlsanity.com/paper/2207.10589/) (2022)
- [SPGroup3D: Superpoint Grouping Network for Indoor 3D Object Detection](https://mlsanity.com/paper/2312.13641/) (2023)
- [X-3D: Explicit 3D Structure Modeling for Point Cloud Recognition](https://mlsanity.com/paper/2404.15010/) (2024)
- [TO-Scene: A Large-scale Dataset for Understanding 3D Tabletop Scenes](https://mlsanity.com/paper/2203.09440/) (2022)

## Abstract (arXiv)

3D object detection in point clouds is a challenging vision task that benefits various applications for understanding the 3D visual world. Lots of recent research focuses on how to exploit end-to-end trainable Hough voting for generating object proposals. However, the current voting strategy can only receive partial votes from the surfaces of potential objects together with severe outlier votes from the cluttered backgrounds, which hampers full utilization of the information from the input point clouds. Inspired by the back-tracing strategy in the conventional Hough voting methods, in this work, we introduce a new 3D object detection method, named as Back-tracing Representative Points Network (BRNet), which generatively back-traces the representative points from the vote centers and also revisits complementary seed points around these generated points, so as to better capture the fine local structural features surrounding the potential objects from the raw point clouds. Therefore, this bottom-up and then top-down strategy in our BRNet enforces mutual consistency between the predicted vote centers and the raw surface points and thus achieves more reliable and flexible object localization and class prediction results. Our BRNet is simple but effective, which significantly outperforms the state-of-the-art methods on two large-scale point cloud datasets, ScanNet V2 (+7.5% in terms of mAP@0.50) and SUN RGB-D (+4.7% in terms of mAP@0.50), while it is still lightweight and efficient. Code will be available at https://github.com/cheng052/BRNet.