mlsanityData to 7 Oct 2026

V-DETR: DETR with Vertex Relative Position Encoding for 3D Object Detection

arXiv 2308.04409 ↗First posted 8 Aug 2023cs.CVcited by 34

Not on the map. Why?

Abstract (arXiv)

We introduce a highly performant 3D object detector for point clouds using the DETR framework. The prior attempts all end up with suboptimal results because they fail to learn accurate inductive biases from the limited scale of training data. In particular, the queries often attend to points that are far away from the target objects, violating the locality principle in object detection. To address the limitation, we introduce a novel 3D Vertex Relative Position Encoding (3DV-RPE) method which computes position encoding for each point based on its relative position to the 3D boxes predicted by the queries in each decoder layer, thus providing clear information to guide the model to focus on points near the objects, in accordance with the principle of locality. In addition, we systematically improve the pipeline from various aspects such as data normalization based on our understanding of the task. We show exceptional results on the challenging ScanNetV2 benchmark, achieving significant improvements over the previous 3DETR in rmAP25/rmAP50 from 65.0%/47.0% to 77.8%/66.0%, respectively. In addition, our method sets a new record on ScanNetV2 and SUN RGB-D datasets.Code will be released at http://github.com/yichaoshen-MS/V-DETR.

Key facts. Cited by 34 papers in the map (data through 7 Oct 2026). Builds on 33 of its 44 references that are in the map. First posted on arXiv 8 Aug 2023.

Builds on

33 of 44 references.

References · 33

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  3. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  4. 2020End-to-End Object Detection with Transformerscited by 7,384
  5. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,410
  6. 2015SSD: Single Shot MultiBox Detectorcited by 3,125
  7. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  8. 2015Fast R-CNNcited by 2,930
  9. 2018PointPillars: Fast Encoders for Object Detection from Point Cloudscited by 1,707
  10. 2017VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detectioncited by 1,323
  11. 2022DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detectioncited by 1,306
  12. 2020Center-based 3D Object Detection and Trackingcited by 1,143
21 more
  1. 2019Deep Hough Voting for 3D Object Detection in Point Cloudscited by 648
  2. 2022DAB-DETR: Dynamic Anchor Boxes are Better Queries for DETRcited by 569
  3. 2022DN-DETR: Accelerate DETR Training by Introducing Query DeNoisingcited by 486
  4. 2021Conditional DETR for Fast Training Convergencecited by 422
  5. 2021An End-to-End Transformer Model for 3D Object Detectioncited by 329
  6. 2021Group-Free 3D Object Detection via Transformerscited by 215
  7. 2021Fast Convergence of DETR with Spatially Modulated Co-Attentioncited by 176
  8. 2022Stratified Transformer for 3D Point Cloud Segmentationcited by 166
  9. 2022DETRs with Hybrid Matchingcited by 149
  10. 20203D-MPA: Multi Proposal Aggregation for 3D Semantic Instance Segmentationcited by 140
  11. 2020H3DNet: 3D Object Detection Using Hybrid Geometric Primitivescited by 123
  12. 2021FCAF3D: Fully Convolutional Anchor-Free 3D Object Detectioncited by 97
  13. 2020MLCVNet: Multi-Level Context VoteNet for 3D Object Detectioncited by 80
  14. 2020Generative Sparse Detection Networks for 3D Single-shot Object Detectioncited by 69
  15. 2021Back-tracing Representative Points for Voting-based 3D Object Detection in Point Cloudscited by 67
  16. 2022CAGroup3D: Class-Aware Grouping for 3D Object Detection on Point Cloudscited by 58
  17. 2022RBGNet: Ray-based Grouping for 3D Object Detectioncited by 40
  18. 2022A Unified Query-based Paradigm for Point Cloud Understandingcited by 32
  19. 2023Super Sparse 3D Object Detectioncited by 32
  20. 2023TR3D: Towards Real-Time Indoor 3D Object Detectioncited by 32
  21. 2022HyperDet3D: Learning a Scene-conditioned 3D Object Detectorcited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 4; 2024: 12; 2025: 8; 2026: 10.

34 papers.

Sort:

By area:

  1. 2023Chat-Scene: Bridging 3D Scene and Large Language Models with Object Identifierscited by 170
  2. 2025SpatialLM: Training Large Language Models for Structured Indoor Modelingcited by 63
  3. 2023Rank-DETR for High Quality Object Detectioncited by 49
  4. 2023Mask4Former: Mask Transformer for 4D Panoptic Segmentationcited by 24
  5. 2024GRA: Detecting Oriented Objects through Group-wise Rotating and Attentioncited by 13
  6. 2024PointSeg: A Training-Free Paradigm for 3D Scene Segmentation via Foundation Modelscited by 9
  7. 2024Diff3DETR:Agent-based Diffusion Model for Semi-supervised 3D Object Detectioncited by 7
  8. 2025Uncertainty Meets Diversity: A Comprehensive Active Learning Framework for Indoor 3D Object Detectioncited by 7
  9. 2024Swin3D++: Effective Multi-Source Pretraining for 3D Indoor Scene Understandingcited by 6
  10. 2025SAS: Segment Any 3D Scene with Integrated 2D Priorscited by 5
  11. 2025State Space Model Meets Transformer: A New Paradigm for 3D Object Detectioncited by 5
  12. 2024Point2Graph: An End-to-end Point Cloud-based 3D Open-Vocabulary Scene Graph for Robot Navigationcited by 4
22 more
  1. 2025DEGround: An Effective Baseline for Ego-centric 3D Visual Grounding with a Homogeneous Frameworkcited by 4
  2. 2025Zoo3D: Zero-Shot 3D Object Detection at Scene Levelcited by 4
  3. 2023Anyview: Generalizable Indoor 3D Object Detection with Variable Framescited by 3
  4. 2024SAM-Guided Masked Token Prediction for 3D Scene Understandingcited by 3
  5. 2024SA3DIP: Segment Any 3D Instance with Potential 3D Priorscited by 3
  6. 2024Training an Open-Vocabulary Monocular 3D Object Detection Model without 3D Datacited by 2
  7. 2026OnlineSI: Taming Large Language Model for Online 3D Understanding and Groundingcited by 2
  8. 2026Few-Shot Incremental 3D Object Detection in Dynamic Indoor Environmentscited by 2
  9. 2024Semantic UV mapping to improve texture inpainting for indoor scenescited by 1
  10. 2024InLUT3D: Challenging real indoor dataset for point cloud analysiscited by 1
  11. 2024MV-DETR: Multi-modality indoor object detection by Multi-View DEtecton TRansformerscited by 1
  12. 2025Fast SceneScript: Fast and Accurate Language-Based 3D Scene Understanding via Multi-Token Predictioncited by 1
  13. 2024Arena: A Patch-of-Interest ViT Inference Acceleration System for Edge-Assisted Video Analyticscited by 0
  14. 2025GraphFusion3D: Dynamic Graph Attention Convolution with Adaptive Cross-Modal Transformer for 3D Object Detectioncited by 0
  15. 2026UniGeo: A Unified 3D Indoor Object Detection Framework Integrating Geometry-Aware Learning and Dynamic Channel Gatingcited by 0
  16. 2026Direction-aware 3D Large Multimodal Modelscited by 0
  17. 2026CT-VDETR: Semi-supervised 3D Trauma Detection in Computed Tomography (CT) scans using Dense Vertex Relative Position Encodingcited by 0
  18. 2026Hyperbolic Distillation: Geometry-Guided Cross-Modal Transfer for Robust 3D Object Detectioncited by 0
  19. 2026PCFootprint: A Large-Scale Dataset and Benchmark for Vectorized Building Footprint Extraction from Aerial LiDAR Point Cloudscited by 0
  20. 2026Breaking the Model Forgetting Cycle in Long-Incremental 3D Object Detectioncited by 0
  21. 2026Geometry Beats Estimated Depth: RGB-Only Multi-Camera 3D Tracking under Sim2Realcited by 0
  22. 2026GraphWrit3R: End-to-End 3D Scene Graph Writingcited by 0