# RTS3D: Real-time Stereo 3D Detection from 4D Feature-Consistency Embedding Space for Autonomous Driving

arXiv: https://arxiv.org/abs/2012.15072 · HTML: https://mlsanity.com/paper/2012.15072/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Dec 2020.

## Built on by (most cited first, 16 of 16)

- [3D Object Detection from Images for Autonomous Driving: A Survey](https://mlsanity.com/paper/2202.02980/) (2022)
- [VPFNet: Improving 3D Object Detection with Virtual Point based LiDAR and Stereo Data Fusion](https://mlsanity.com/paper/2111.14382/) (2021)
- [MonoCD: Monocular 3D Object Detection with Complementary Depths](https://mlsanity.com/paper/2404.03181/) (2024)
- [MonoDGP: Monocular 3D Object Detection with Decoupled-Query and Geometry-Error Priors](https://mlsanity.com/paper/2410.19590/) (2024)
- [RS2G: Data-Driven Scene-Graph Extraction and Embedding for Robust Autonomous Perception and Scenario Understanding](https://mlsanity.com/paper/2304.08600/) (2023)
- [ESGN: Efficient Stereo Geometry Network for Fast 3D Object Detection](https://mlsanity.com/paper/2111.14055/) (2021)
- [Stereo Neural Vernier Caliper](https://mlsanity.com/paper/2203.11018/) (2022)
- [An Empirical Study of Pseudo-Labeling for Image-based 3D Object Detection](https://mlsanity.com/paper/2208.07137/) (2022)
- [Transformer-based stereo-aware 3D object detection from binocular images](https://mlsanity.com/paper/2304.11906/) (2023)
- [StereoDETR: Stereo-based Transformer for 3D Object Detection](https://mlsanity.com/paper/2511.18788/) (2025)
- [Joint stereo 3D object detection and implicit surface reconstruction](https://mlsanity.com/paper/2111.12924/) (2021)
- [Accurate and Real-time Pseudo Lidar Detection: Is Stereo Neural Network Really Necessary?](https://mlsanity.com/paper/2206.13858/) (2022)
- [MaDis-Stereo: Enhanced Stereo Matching via Distilled Masked Image Modeling](https://mlsanity.com/paper/2409.02846/) (2024)
- [DDStereo: Efficient Dual Decoder Transformers for Stereo 3D Road Anomaly Detection](https://mlsanity.com/paper/2606.24805/) (2026)
- [Shape Prior Non-Uniform Sampling Guided Real-time Stereo 3D Object Detection](https://mlsanity.com/paper/2106.10013/) (2021)
- [Entropy Loss: An Interpretability Amplifier of 3D Object Detection Network for Intelligent Driving](https://mlsanity.com/paper/2409.00839/) (2024)

## Abstract (arXiv)

Although the recent image-based 3D object detection methods using Pseudo-LiDAR representation have shown great capabilities, a notable gap in efficiency and accuracy still exist compared with LiDAR-based methods. Besides, over-reliance on the stand-alone depth estimator, requiring a large number of pixel-wise annotations in the training stage and more computation in the inferencing stage, limits the scaling application in the real world. In this paper, we propose an efficient and accurate 3D object detection method from stereo images, named RTS3D. Different from the 3D occupancy space in the Pseudo-LiDAR similar methods, we design a novel 4D feature-consistent embedding (FCE) space as the intermediate representation of the 3D scene without depth supervision. The FCE space encodes the object's structural and semantic information by exploring the multi-scale feature consistency warped from stereo pair. Furthermore, a semantic-guided RBF (Radial Basis Function) and a structure-aware attention module are devised to reduce the influence of FCE space noise without instance mask supervision. Experiments on the KITTI benchmark show that RTS3D is the first true real-time system (FPS>24) for stereo image 3D detection meanwhile achieves 10% improvement in average precision comparing with the previous state-of-the-art method. The code will be available at https://github.com/Banconxuan/RTS3D