mlsanityData to 7 Oct 2026

Revisiting 3D Object Detection From an Egocentric Perspective

arXiv 2112.07787 ↗First posted 14 Dec 2021cs.CV cs.ROcited by 18

Not on the map. Why?

Abstract (arXiv)

3D object detection is a key module for safety-critical robotics applications such as autonomous driving. For these applications, we care most about how the detections affect the ego-agent's behavior and safety (the egocentric perspective). Intuitively, we seek more accurate descriptions of object geometry when it's more likely to interfere with the ego-agent's motion trajectory. However, current detection metrics, based on box Intersection-over-Union (IoU), are object-centric and aren't designed to capture the spatio-temporal relationship between objects and the ego-agent. To address this issue, we propose a new egocentric measure to evaluate 3D object detection, namely Support Distance Error (SDE). Our analysis based on SDE reveals that the egocentric detection quality is bounded by the coarse geometry of the bounding boxes. Given the insight that SDE would benefit from more accurate geometry descriptions, we propose to represent objects as amodal contours, specifically amodal star-shaped polygons, and devise a simple model, StarPoly, to predict such contours. Our experiments on the large-scale Waymo Open Dataset show that SDE better reflects the impact of detection quality on the ego-agent's safety compared to IoU; and the estimated contours from StarPoly consistently improve the egocentric detection quality over recent 3D object detectors.

Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). Builds on 51 of its 69 references that are in the map. First posted on arXiv 14 Dec 2021.

Builds on

51 of 69 references.

References · 51

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  3. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  4. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,239
  5. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,436
  6. 2018PointPillars: Fast Encoders for Object Detection from Point Cloudscited by 1,707
  7. 2017VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detectioncited by 1,323
  8. 2018PointRCNN: 3D Object Proposal Generation and Detection from Point Cloudcited by 939
  9. 2019PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detectioncited by 890
  10. 2016Multi-View 3D Object Detection Network for Autonomous Drivingcited by 721
  11. 2019Deep Hough Voting for 3D Object Detection in Point Cloudscited by 648
  12. 2017Frustum PointNets for 3D Object Detection from RGB-D Datacited by 640
38 more
  1. 20203DSSD: Point-based 3D Single Stage Object Detectorcited by 440
  2. 2019PIXOR: Real-time 3D Object Detection from Point Cloudscited by 358
  3. 2020Point-GNN: Graph Neural Network for 3D Object Detection in a Point Cloudcited by 335
  4. 2017Joint 3D Proposal Generation and Object Detection from View Aggregationcited by 325
  5. 2019STD: Sparse-to-Dense 3D Object Detector for Point Cloudcited by 282
  6. 2020Understanding Human Hands in Contact at Internet Scalecited by 281
  7. 2020Deep Continuous Fusion for Multi-Sensor 3D Object Detectioncited by 263
  8. 2020Fast and Furious: Real Time End-to-End 3D Detection, Tracking and Motion Forecasting with a Single Convolutional Netcited by 228
  9. 2019EPIC-Fusion: Audio-Visual Temporal Binding for Egocentric Action Recognitioncited by 226
  10. 2016Shape Completion using 3D-Encoder-Predictor CNNs and Shape Synthesiscited by 219
  11. 2020Multi-Task Multi-Sensor Fusion for 3D Object Detectioncited by 203
  12. 2021Boundary IoU: Improving Object-Centric Image Segmentation Evaluationcited by 186
  13. 2019End-to-End Multi-View Fusion for 3D Object Detection in LiDAR Point Cloudscited by 171
  14. 2017PointFusion: Deep Sensor Fusion for 3D Bounding Box Estimationcited by 170
  15. 2019MVX-Net: Multimodal VoxelNet for 3D Object Detectioncited by 162
  16. 2021Offboard 3D Object Detection from Point Cloud Sequencescited by 144
  17. 2020Pillar-based Object Detection for Autonomous Drivingcited by 137
  18. 2019Fast Point R-CNNcited by 119
  19. 2020ImVoteNet: Boosting 3D Object Detection in Point Clouds with Image Votescited by 119
  20. 2015Deep Sliding Shapes for Amodal 3D Object Detection in RGB-D Imagescited by 111
  21. 2019Ego-Pose Estimation and Forecasting as Real-Time PD Controlcited by 87
  22. 2019You2Me: Inferring Body Pose in Egocentric Video via First and Second Person Interactionscited by 82
  23. 2020HVNet: Hybrid Voxel Network for LiDAR Based 3D Object Detectioncited by 82
  24. 2019xR-EgoPose: Egocentric 3D Human Pose from an HMD Cameracited by 78
  25. 2020HDNET: Exploiting HD Maps for 3D Object Detectioncited by 78
  26. 2019Gate-Shift Networks for Video Action Recognitioncited by 76
  27. 2016Vote3Deep: Fast Object Detection in 3D Point Clouds Using Efficient Convolutional Neural Networkscited by 72
  28. 2019What You See is What You Get: Exploiting Visibility for 3D Object Detectioncited by 69
  29. 2020LiDAR-based Online 3D Video Object Detection with Graph-based Message Passing and Spatiotemporal Transformer Attentioncited by 65
  30. 20163D Fully Convolutional Network for Vehicle Detection in Point Cloudcited by 57
  31. 2018LSTA: Long Short-Term Attention for Egocentric Action Recognitioncited by 57
  32. 2018Complex-YOLO: Real-time 3D Object Detection on Point Cloudscited by 56
  33. 2020Learning to Evaluate Perception Models Using Planner-Centric Metricscited by 49
  34. 2016Going Deeper into First-Person Activity Recognitioncited by 48
  35. 2019Sensor Fusion for Joint 3D Object Detection and Semantic Segmentationcited by 48
  36. 2018IPOD: Intensive Point-based Object Detector for Point Cloudcited by 46
  37. 2019StarNet: Targeted Computation for Object Detection in Point Cloudscited by 46
  38. 2020An LSTM Approach to Temporal 3D Object Detection in LiDAR Point Cloudscited by 45

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 6; 2023: 6; 2024: 5; 2025: 0; 2026: 1.

18 papers.

Sort:

By area:

  1. 20223D Object Detection for Autonomous Driving: A Comprehensive Surveycited by 132
  2. 20223D Object Detection from Images for Autonomous Driving: A Surveycited by 66
  3. 2022Motion Inspired Unsupervised Perception and Prediction in Autonomous Drivingcited by 59
  4. 2023Revisiting Domain-Adaptive 3D Object Detection by Reliable, Diverse and Class-balanced Pseudo-Labelingcited by 28
  5. 2023Exploring Active 3D Object Detection from a Generalization Perspectivecited by 26
  6. 2024Find n' Propagate: Open-Vocabulary 3D Object Detection in Urban Environmentscited by 21
  7. 2023On Offline Evaluation of 3D Object Detection for Autonomous Drivingcited by 11
  8. 2023Transcendental Idealism of Planner: Evaluating Perception from Planning Perspective for Autonomous Drivingcited by 9
  9. 2024MOS: Model Synergy for Test-Time Adaptation on LiDAR-Based 3D Object Detectioncited by 6
  10. 2022CG-SSD: Corner Guided Single Stage 3D Object Detection from LiDAR Point Cloudcited by 5
  11. 2022USC: Uncompromising Spatial Constraints for Safety-Oriented 3D Object Detectors in Autonomous Drivingcited by 5
  12. 2023SURE-Val: Safe Urban Relevance Extension and Validationcited by 3
6 more
  1. 2024Instance-Level Safety-Aware Fidelity of Synthetic Data and Its Calibrationcited by 2
  2. 2024EC-IoU: Orienting Safety for Object Detectors via Ego-Centric Intersection-over-Unioncited by 2
  3. 2022You Only Label Once: 3D Box Adaptation from Point Cloud to Image via Semi-Supervised Learningcited by 1
  4. 2024STONE: A Submodular Optimization Framework for Active 3D Object Detectioncited by 1
  5. 2023Open-CRB: Towards Open World Active Learning for 3D Object Detectioncited by 0
  6. 2026Safety-Aligned 3D Object Detection: Single-Vehicle, Cooperative, and End-to-End Perspectivescited by 0