# Real-Time 3D Object Detection with Inference-Aligned Learning

arXiv: https://arxiv.org/abs/2511.16140 · HTML: https://mlsanity.com/paper/2511.16140/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Detection and dense prediction distillation, one of 12 directions in Knowledge distillation.
- Builds on 32 of its 44 references that are in the map.
- First posted on arXiv 20 Nov 2025.

## Placed in

- Knowledge distillation / [Detection and dense prediction distillation](https://mlsanity.com/distill/detection-and-dense-prediction-distillation/)

## Builds on (most cited first, 30 of 32)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [FCOS: Fully Convolutional One-Stage Object Detection](https://mlsanity.com/paper/1904.01355/) (2019)
- [MMDetection: Open MMLab Detection Toolbox and Benchmark](https://mlsanity.com/paper/1906.07155/) (2019)
- [Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regression](https://mlsanity.com/paper/1902.09630/) (2019)
- [4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks](https://mlsanity.com/paper/1904.08755/) (2019)
- [YOLOX: Exceeding YOLO Series in 2021](https://mlsanity.com/paper/2107.08430/) (2021)
- [Deep Hough Voting for 3D Object Detection in Point Clouds](https://mlsanity.com/paper/1904.09664/) (2019)
- [Born Again Neural Networks](https://mlsanity.com/paper/1805.04770/) (2018)
- [Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self Distillation](https://mlsanity.com/paper/1905.08094/) (2019)
- [Distance-IoU Loss: Faster and Better Learning for Bounding Box Regression](https://mlsanity.com/paper/1911.08287/) (2019)
- [Generalized Focal Loss: Learning Qualified and Distributed Bounding Boxes for Dense Object Detection](https://mlsanity.com/paper/2006.04388/) (2020)
- [An End-to-End Transformer Model for 3D Object Detection](https://mlsanity.com/paper/2109.08141/) (2021)
- [VarifocalNet: An IoU-aware Dense Object Detector](https://mlsanity.com/paper/2008.13367/) (2020)
- [Group-Free 3D Object Detection via Transformers](https://mlsanity.com/paper/2104.00678/) (2021)
- [UnitBox: An Advanced Object Detection Network](https://mlsanity.com/paper/1608.01471/) (2016)
- [Knowledge Distillation by On-the-Fly Native Ensemble](https://mlsanity.com/paper/1806.04606/) (2018)
- [OTA: Optimal Transport Assignment for Object Detection](https://mlsanity.com/paper/2103.14259/) (2021)
- [Probabilistic Anchor Assignment with IoU Prediction for Object Detection](https://mlsanity.com/paper/2007.08103/) (2020)
- [Regularizing Class-wise Predictions via Self-knowledge Distillation](https://mlsanity.com/paper/2003.13964/) (2020)
- [AutoAssign: Differentiable Label Assignment for Dense Object Detection](https://mlsanity.com/paper/2007.03496/) (2020)
- [FreeAnchor: Learning to Match Anchors for Visual Object Detection](https://mlsanity.com/paper/1909.02466/) (2019)
- [MLCVNet: Multi-Level Context VoteNet for 3D Object Detection](https://mlsanity.com/paper/2004.05679/) (2020)
- [Generative Sparse Detection Networks for 3D Single-shot Object Detection](https://mlsanity.com/paper/2006.12356/) (2020)
- [DAMO-YOLO : A Report on Real-Time Object Detection Design](https://mlsanity.com/paper/2211.15444/) (2022)
- [Learning from Noisy Anchors for One-stage Object Detection](https://mlsanity.com/paper/1912.05086/) (2019)
- [Multiple Anchor Learning for Visual Object Detection](https://mlsanity.com/paper/1912.02252/) (2019)
- [LGD: Label-guided Self-distillation for Object Detection](https://mlsanity.com/paper/2109.11496/) (2021)
- [Point-GCC: Universal Self-supervised 3D Scene Pre-training via Geometry-Color Contrast](https://mlsanity.com/paper/2305.19623/) (2023)

## Abstract (arXiv)

Real-time 3D object detection from point clouds is essential for dynamic scene understanding in applications such as augmented reality, robotics and navigation. We introduce a novel Spatial-prioritized and Rank-aware 3D object detection (SR3D) framework for indoor point clouds, to bridge the gap between how detectors are trained and how they are evaluated. This gap stems from the lack of spatial reliability and ranking awareness during training, which conflicts with the ranking-based prediction selection used as inference. Such a training-inference gap hampers the model's ability to learn representations aligned with inference-time behavior. To address the limitation, SR3D consists of two components tailored to the spatial nature of point clouds during training: a novel spatial-prioritized optimal transport assignment that dynamically emphasizes well-located and spatially reliable samples, and a rank-aware adaptive self-distillation scheme that adaptively injects ranking perception via a self-distillation paradigm. Extensive experiments on ScanNet V2 and SUN RGB-D show that SR3D effectively bridges the training-inference gap and significantly outperforms prior methods in accuracy while maintaining real-time speed.