# Rethinking IoU-based Optimization for Single-stage 3D Object Detection

arXiv: https://arxiv.org/abs/2207.09332 · HTML: https://mlsanity.com/paper/2207.09332/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 43 references that are in the map.
- First posted on arXiv 19 Jul 2022.

## Builds on (most cited first, 30 of 35)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [PointPillars: Fast Encoders for Object Detection from Point Clouds](https://mlsanity.com/paper/1812.05784/) (2018)
- [Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regression](https://mlsanity.com/paper/1902.09630/) (2019)
- [VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection](https://mlsanity.com/paper/1711.06396/) (2017)
- [Center-based 3D Object Detection and Tracking](https://mlsanity.com/paper/2006.11275/) (2020)
- [PointRCNN: 3D Object Proposal Generation and Detection from Point Cloud](https://mlsanity.com/paper/1812.04244/) (2018)
- [PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detection](https://mlsanity.com/paper/1912.13192/) (2019)
- [Multi-View 3D Object Detection Network for Autonomous Driving](https://mlsanity.com/paper/1611.07759/) (2016)
- [Frustum PointNets for 3D Object Detection from RGB-D Data](https://mlsanity.com/paper/1711.08488/) (2017)
- [Distance-IoU Loss: Faster and Better Learning for Bounding Box Regression](https://mlsanity.com/paper/1911.08287/) (2019)
- [3DSSD: Point-based 3D Single Stage Object Detector](https://mlsanity.com/paper/2002.10187/) (2020)
- [Voxel R-CNN: Towards High Performance Voxel-based 3D Object Detection](https://mlsanity.com/paper/2012.15712/) (2020)
- [Generalized Focal Loss: Learning Qualified and Distributed Bounding Boxes for Dense Object Detection](https://mlsanity.com/paper/2006.04388/) (2020)
- [From Points to Parts: 3D Object Detection from Point Cloud with Part-aware and Part-aggregation Network](https://mlsanity.com/paper/1907.03670/) (2019)
- [Point-GNN: Graph Neural Network for 3D Object Detection in a Point Cloud](https://mlsanity.com/paper/2003.01251/) (2020)
- [STD: Sparse-to-Dense 3D Object Detector for Point Cloud](https://mlsanity.com/paper/1907.10471/) (2019)
- [Voxel Transformer for 3D Object Detection](https://mlsanity.com/paper/2109.02497/) (2021)
- [Submanifold Sparse Convolutional Networks](https://mlsanity.com/paper/1706.01307/) (2017)
- [End-to-End Multi-View Fusion for 3D Object Detection in LiDAR Point Clouds](https://mlsanity.com/paper/1910.06528/) (2019)
- [Rethinking Rotated Object Detection with Gaussian Wasserstein Distance Loss](https://mlsanity.com/paper/2101.11952/) (2021)
- [Pillar-based Object Detection for Autonomous Driving](https://mlsanity.com/paper/2007.10323/) (2020)
- [LiDAR R-CNN: An Efficient and Universal 3D Object Detector](https://mlsanity.com/paper/2103.15297/) (2021)
- [IoU Loss for 2D/3D Object Detection](https://mlsanity.com/paper/1908.03851/) (2019)
- [SE-SSD: Self-Ensembling Single-Stage Object Detector From Point Cloud](https://mlsanity.com/paper/2104.09804/) (2021)
- [Improving 3D Object Detection with Channel-wise Transformer](https://mlsanity.com/paper/2108.10723/) (2021)

## Built on by (most cited first, 19 of 19)

- [Dual Radar: A Multi-modal Dataset with Dual 4D Radar for Autonomous Driving](https://mlsanity.com/paper/2310.07602/) (2023)
- [Uni3DETR: Unified 3D Detection Transformer](https://mlsanity.com/paper/2310.05699/) (2023)
- [V2X-Radar: A Multi-modal Dataset with 4D Radar for Cooperative Perception](https://mlsanity.com/paper/2411.10962/) (2024)
- [Unlocking Textual and Visual Wisdom: Open-Vocabulary 3D Object Detection Enhanced by Comprehensive Guidance from Text and Image](https://mlsanity.com/paper/2407.05256/) (2024)
- [CT3D++: Improving 3D Object Detection with Keypoint-induced Channel-wise Transformer](https://mlsanity.com/paper/2406.08152/) (2024)
- [Uncertainty Meets Diversity: A Comprehensive Active Learning Framework for Indoor 3D Object Detection](https://mlsanity.com/paper/2503.16125/) (2025)
- [CORENet: Cross-Modal 4D Radar Denoising Network with LiDAR Supervision for Autonomous Driving](https://mlsanity.com/paper/2508.13485/) (2025)
- [Polarimetric Imaging for Perception](https://mlsanity.com/paper/2305.14787/) (2023)
- [CCF: Complementary Collaborative Fusion for Domain Generalized Multi-Modal 3D Object Detection](https://mlsanity.com/paper/2603.23276/) (2026)
- [MonoTDP: Twin Depth Perception for Monocular 3D Object Detection in Adverse Scenes](https://mlsanity.com/paper/2305.10974/) (2023)
- [DCDet: Dynamic Cross-based 3D Object Detector](https://mlsanity.com/paper/2401.07240/) (2024)
- [All You Need for Object Detection: From Pixels, Points, and Prompts to Next-Gen Fusion and Multimodal LLMs/VLMs in Autonomous Vehicles](https://mlsanity.com/paper/2510.26641/) (2025)
- [Leveraging Anchor-based LiDAR 3D Object Detection via Point Assisted Sample Selection](https://mlsanity.com/paper/2403.01978/) (2024)
- [SGCCNet: Single-Stage 3D Object Detector With Saliency-Guided Data Augmentation and Confidence Correction Mechanism](https://mlsanity.com/paper/2407.01239/) (2024)
- [3DPillars: Pillar-based two-stage 3D object detection](https://mlsanity.com/paper/2509.05780/) (2025)
- [TransBridge: Boost 3D Object Detection by Scene-Level Completion with Transformer Decoder](https://mlsanity.com/paper/2512.11926/) (2025)
- [Point Virtual Transformer](https://mlsanity.com/paper/2602.06406/) (2026)
- [Geometry Meets Semantics: Fractional Gradient Stabilization for Semantic-Driven Bounding Box Optimization in Visual Detection Tasks](https://mlsanity.com/paper/2607.23530/) (2026)
- [PGL-3D: Towards Progressive Geometric Learning for 3D Visual Query Localization](https://mlsanity.com/paper/2609.33558/) (2026)

## Abstract (arXiv)

Since Intersection-over-Union (IoU) based optimization maintains the consistency of the final IoU prediction metric and losses, it has been widely used in both regression and classification branches of single-stage 2D object detectors. Recently, several 3D object detection methods adopt IoU-based optimization and directly replace the 2D IoU with 3D IoU. However, such a direct computation in 3D is very costly due to the complex implementation and inefficient backward operations. Moreover, 3D IoU-based optimization is sub-optimal as it is sensitive to rotation and thus can cause training instability and detection performance deterioration. In this paper, we propose a novel Rotation-Decoupled IoU (RDIoU) method that can mitigate the rotation-sensitivity issue, and produce more efficient optimization objectives compared with 3D IoU during the training stage. Specifically, our RDIoU simplifies the complex interactions of regression parameters by decoupling the rotation variable as an independent term, yet preserving the geometry of 3D IoU. By incorporating RDIoU into both the regression and classification branches, the network is encouraged to learn more precise bounding boxes and concurrently overcome the misalignment issue between classification and regression. Extensive experiments on the benchmark KITTI and Waymo Open Dataset validate that our RDIoU method can bring substantial improvement for the single-stage 3D object detection.