# TR3D: Towards Real-Time Indoor 3D Object Detection

arXiv: https://arxiv.org/abs/2302.02858 · HTML: https://mlsanity.com/paper/2302.02858/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 23 references that are in the map.
- First posted on arXiv 6 Feb 2023.

## Builds on (most cited first, 17 of 17)

- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [Deep Hough Voting for 3D Object Detection in Point Clouds](https://mlsanity.com/paper/1904.09664/) (2019)
- [An End-to-End Transformer Model for 3D Object Detection](https://mlsanity.com/paper/2109.08141/) (2021)
- [3D-SIS: 3D Semantic Instance Segmentation of RGB-D Scans](https://mlsanity.com/paper/1812.07003/) (2018)
- [Group-Free 3D Object Detection via Transformers](https://mlsanity.com/paper/2104.00678/) (2021)
- [EPNet: Enhancing Point Features with Image Semantics for 3D Object Detection](https://mlsanity.com/paper/2007.08856/) (2020)
- [ImVoxelNet: Image to Voxels Projection for Monocular and Multi-View General-Purpose 3D Object Detection](https://mlsanity.com/paper/2106.01178/) (2021)
- [Multimodal Token Fusion for Vision Transformers](https://mlsanity.com/paper/2204.08721/) (2022)
- [H3DNet: 3D Object Detection Using Hybrid Geometric Primitives](https://mlsanity.com/paper/2006.05682/) (2020)
- [ImVoteNet: Boosting 3D Object Detection in Point Clouds with Image Votes](https://mlsanity.com/paper/2001.10692/) (2020)
- [FCAF3D: Fully Convolutional Anchor-Free 3D Object Detection](https://mlsanity.com/paper/2112.00322/) (2021)
- [Generative Sparse Detection Networks for 3D Single-shot Object Detection](https://mlsanity.com/paper/2006.12356/) (2020)
- [Back-tracing Representative Points for Voting-based 3D Object Detection in Point Clouds](https://mlsanity.com/paper/2104.06114/) (2021)
- [EPNet++: Cascade Bi-directional Fusion for Multi-Modal 3D Object Detection](https://mlsanity.com/paper/2112.11088/) (2021)
- [RBGNet: Ray-based Grouping for 3D Object Detection](https://mlsanity.com/paper/2204.02251/) (2022)
- [HyperDet3D: Learning a Scene-conditioned 3D Object Detector](https://mlsanity.com/paper/2204.05599/) (2022)
- [Multi-Modality Task Cascade for 3D Object Detection](https://mlsanity.com/paper/2107.04013/) (2021)

## Built on by (most cited first, 30 of 32)

- [OneFormer3D: One Transformer for Unified Point Cloud Segmentation](https://mlsanity.com/paper/2311.14405/) (2023)
- [EmbodiedSAM: Online Segment Any 3D Thing in Real Time](https://mlsanity.com/paper/2408.11811/) (2024)
- [Cubify Anything: Scaling Indoor 3D Object Detection](https://mlsanity.com/paper/2412.04458/) (2024)
- [V-DETR: DETR with Vertex Relative Position Encoding for 3D Object Detection](https://mlsanity.com/paper/2308.04409/) (2023)
- [Uni3DETR: Unified 3D Detection Transformer](https://mlsanity.com/paper/2310.05699/) (2023)
- [TSP3D: Text-guided Sparse Voxel Pruning for Efficient 3D Visual Grounding](https://mlsanity.com/paper/2502.10392/) (2025)
- [Point-GCC: Universal Self-supervised 3D Scene Pre-training via Geometry-Color Contrast](https://mlsanity.com/paper/2305.19623/) (2023)
- [Memory-based Adapters for Online 3D Scene Perception](https://mlsanity.com/paper/2403.06974/) (2024)
- [FastCAD: Real-Time CAD Retrieval and Alignment from Scans and Videos](https://mlsanity.com/paper/2403.15161/) (2024)
- [Boxer: Robust Lifting of Open-World 2D Bounding Boxes to 3D](https://mlsanity.com/paper/2604.05212/) (2026)
- [State Space Model Meets Transformer: A New Paradigm for 3D Object Detection](https://mlsanity.com/paper/2503.14493/) (2025)
- [GO-N3RDet: Geometry Optimized NeRF-enhanced 3D Object Detector](https://mlsanity.com/paper/2503.15211/) (2025)
- [3D Small Object Detection with Dynamic Spatial Pruning](https://mlsanity.com/paper/2305.03716/) (2023)
- [Zoo3D: Zero-Shot 3D Object Detection at Scene Level](https://mlsanity.com/paper/2511.20253/) (2025)
- [Object Dynamics Modeling with Hierarchical Point Cloud-based Representations](https://mlsanity.com/paper/2404.06044/) (2024)
- [BoxFusion: Reconstruction-Free Open-Vocabulary 3D Object Detection via Real-Time Multi-View Box Fusion](https://mlsanity.com/paper/2506.15610/) (2025)
- [Progressive Classifier and Feature Extractor Adaptation for Unsupervised Domain Adaptation on Point Clouds](https://mlsanity.com/paper/2311.16474/) (2023)
- [Towards 3D Objectness Learning in an Open World](https://mlsanity.com/paper/2510.17686/) (2025)
- [Few-Shot Incremental 3D Object Detection in Dynamic Indoor Environments](https://mlsanity.com/paper/2604.07997/) (2026)
- [Care3D: An Active 3D Object Detection Dataset of Real Robotic-Care Environments](https://mlsanity.com/paper/2310.05600/) (2023)
- [InLUT3D: Challenging real indoor dataset for point cloud analysis](https://mlsanity.com/paper/2408.03338/) (2024)
- [TUN3D: Towards Real-World Scene Understanding from Unposed Images](https://mlsanity.com/paper/2509.21388/) (2025)
- [Fast SceneScript: Fast and Accurate Language-Based 3D Scene Understanding via Multi-Token Prediction](https://mlsanity.com/paper/2512.05597/) (2025)
- [Map-Det3D: Metric Feed-Forward 3D Reconstruction Prior for Multi-view 3D Object Detection from Streaming Inputs](https://mlsanity.com/paper/2608.12179/) (2026)
- [DetAny4D: Detect Anything 4D Temporally in a Streaming RGB Video](https://mlsanity.com/paper/2511.18814/) (2025)
- [GraphFusion3D: Dynamic Graph Attention Convolution with Adaptive Cross-Modal Transformer for 3D Object Detection](https://mlsanity.com/paper/2512.02991/) (2025)
- [Enhancing annotations for 5D apple pose estimation through 3D Gaussian Splatting (3DGS)](https://mlsanity.com/paper/2512.20148/) (2025)
- [UniGeo: A Unified 3D Indoor Object Detection Framework Integrating Geometry-Aware Learning and Dynamic Channel Gating](https://mlsanity.com/paper/2601.22616/) (2026)
- [Group3D: MLLM-Driven Semantic Grouping for Open-Vocabulary 3D Object Detection](https://mlsanity.com/paper/2603.21944/) (2026)
- [Hyperbolic Distillation: Geometry-Guided Cross-Modal Transfer for Robust 3D Object Detection](https://mlsanity.com/paper/2605.09899/) (2026)

## Abstract (arXiv)

Recently, sparse 3D convolutions have changed 3D object detection. Performing on par with the voting-based approaches, 3D CNNs are memory-efficient and scale to large scenes better. However, there is still room for improvement. With a conscious, practice-oriented approach to problem-solving, we analyze the performance of such methods and localize the weaknesses. Applying modifications that resolve the found issues one by one, we end up with TR3D: a fast fully-convolutional 3D object detection model trained end-to-end, that achieves state-of-the-art results on the standard benchmarks, ScanNet v2, SUN RGB-D, and S3DIS. Moreover, to take advantage of both point cloud and RGB inputs, we introduce an early fusion of 2D and 3D features. We employ our fusion module to make conventional 3D object detection methods multimodal and demonstrate an impressive boost in performance. Our model with early feature fusion, which we refer to as TR3D+FF, outperforms existing 3D object detection approaches on the SUN RGB-D dataset. Overall, besides being accurate, both TR3D and TR3D+FF models are lightweight, memory-efficient, and fast, thereby marking another milestone on the way toward real-time 3D object detection. Code is available at https://github.com/SamsungLabs/tr3d .