# Scatter Points in Space: 3D Detection from Multi-view Monocular Images

arXiv: https://arxiv.org/abs/2208.14738 · HTML: https://mlsanity.com/paper/2208.14738/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 20 references that are in the map.
- First posted on arXiv 31 Aug 2022.

## Builds on (most cited first, 19 of 19)

- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [MVSNet: Depth Inference for Unstructured Multi-view Stereo](https://mlsanity.com/paper/1804.02505/) (2018)
- [Deep Ordinal Regression Network for Monocular Depth Estimation](https://mlsanity.com/paper/1806.02446/) (2018)
- [Deep Hough Voting for 3D Object Detection in Point Clouds](https://mlsanity.com/paper/1904.09664/) (2019)
- [PointPainting: Sequential Fusion for 3D Object Detection](https://mlsanity.com/paper/1911.10150/) (2019)
- [Categorical Depth Distribution Network for Monocular 3D Object Detection](https://mlsanity.com/paper/2103.01100/) (2021)
- [Atlas: End-to-End 3D Scene Reconstruction from Posed Images](https://mlsanity.com/paper/2003.10432/) (2020)
- [Total3DUnderstanding: Joint Layout, Object Pose and Mesh Reconstruction for Indoor Scenes from a Single Image](https://mlsanity.com/paper/2002.12212/) (2020)
- [Point-Based Multi-View Stereo Network](https://mlsanity.com/paper/1908.04422/) (2019)
- [Scan2CAD: Learning CAD Model Alignment in RGB-D Scans](https://mlsanity.com/paper/1811.11187/) (2018)
- [Deep Mesh Reconstruction from Single RGB Images via Topology Modification Networks](https://mlsanity.com/paper/1909.00321/) (2019)
- [TransformerFusion: Monocular RGB Scene Reconstruction using Transformers](https://mlsanity.com/paper/2107.02191/) (2021)
- [QuadricSLAM: Dual Quadrics from Object Detections as Landmarks in Object-oriented SLAM](https://mlsanity.com/paper/1804.04011/) (2018)
- [FroDO: From Detections to 3D Objects](https://mlsanity.com/paper/2005.05125/) (2020)
- [Track to Reconstruct and Reconstruct to Track](https://mlsanity.com/paper/1910.00130/) (2019)
- [Voxel-based 3D Detection and Reconstruction of Multiple Objects from a Single Image](https://mlsanity.com/paper/2111.03098/) (2021)
- [MOLTR: Multiple Object Localisation, Tracking, and Reconstruction from Monocular RGB Videos](https://mlsanity.com/paper/2012.05360/) (2020)

## Abstract (arXiv)

3D object detection from monocular image(s) is a challenging and long-standing problem of computer vision. To combine information from different perspectives without troublesome 2D instance tracking, recent methods tend to aggregate multiview feature by sampling regular 3D grid densely in space, which is inefficient. In this paper, we attempt to improve multi-view feature aggregation by proposing a learnable keypoints sampling method, which scatters pseudo surface points in 3D space, in order to keep data sparsity. The scattered points augmented by multi-view geometric constraints and visual features are then employed to infer objects location and shape in the scene. To make up the limitations of single frame and model multi-view geometry explicitly, we further propose a surface filter module for noise suppression. Experimental results show that our method achieves significantly better performance than previous works in terms of 3D detection (more than 0.1 AP improvement on some categories of ScanNet). The code will be publicly available.