mlsanityData to 7 Oct 2026

SM3D: Simultaneous Monocular Mapping and 3D Detection

arXiv 2111.12643 ↗First posted 24 Nov 2021cs.CVcited by 3

Not on the map. Why?

Abstract (arXiv)

Mapping and 3D detection are two major issues in vision-based robotics, and self-driving. While previous works only focus on each task separately, we present an innovative and efficient multi-task deep learning framework (SM3D) for Simultaneous Mapping and 3D Detection by bridging the gap with robust depth estimation and "Pseudo-LiDAR" point cloud for the first time. The Mapping module takes consecutive monocular frames to generate depth and pose estimation. In 3D Detection module, the depth estimation is projected into 3D space to generate "Pseudo-LiDAR" point cloud, where LiDAR-based 3D detector can be leveraged on point cloud for vehicular 3D detection and localization. By end-to-end training of both modules, the proposed mapping and 3D detection method outperforms the state-of-the-art baseline by 10.0% and 13.2% in accuracy, respectively. While achieving better accuracy, our monocular multi-task SM3D is more than 2 times faster than pure stereo 3D detector, and 18.3% faster than using two modules separately.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 22 of its 27 references that are in the map. First posted on arXiv 24 Nov 2021.

Builds on

22 of 27 references.

References · 22

  1. 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,678
  2. 2018PointPillars: Fast Encoders for Object Detection from Point Cloudscited by 1,707
  3. 2017SuperPoint: Self-Supervised Interest Point Detection and Descriptioncited by 1,313
  4. 2015ORB-SLAM: a Versatile and Accurate Monocular SLAM Systemcited by 983
  5. 2018PointRCNN: 3D Object Proposal Generation and Detection from Point Cloudcited by 939
  6. 2017Unsupervised Learning of Depth and Ego-Motion from Videocited by 778
  7. 2018Pseudo-LiDAR from Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Drivingcited by 455
  8. 2019From Big to Small: Multi-Scale Local Planar Guidance for Monocular Depth Estimationcited by 379
  9. 2018GeoNet: Unsupervised Learning of Dense Depth, Optical Flow and Camera Posecited by 341
  10. 20163D Bounding Box Estimation Using Deep Learning and Geometrycited by 277
  11. 2019Pseudo-LiDAR++: Accurate Depth for 3D Object Detection in Autonomous Drivingcited by 188
  12. 2018DF-Net: Unsupervised Joint Learning of Depth and Flow using Cross-Task Consistencycited by 148
10 more
  1. 2018ROI-10D: Monocular Lifting of 2D Detection to 6D Pose and Metric Shapecited by 123
  2. 2019Fast Point R-CNNcited by 119
  3. 2019Monocular 3D Object Detection with Pseudo-LiDAR Point Cloudcited by 113
  4. 2018MonoGRNet: A Geometric Reasoning Network for Monocular 3D Object Localizationcited by 84
  5. 2020End-to-End Pseudo-LiDAR for Image-Based 3D Object Detectioncited by 80
  6. 20163D Object Proposals using Stereo Imagery for Accurate Object Class Detectioncited by 58
  7. 2018Learning Rigidity in Dynamic Scenes with a Moving Camera for 3D Motion Field Estimationcited by 38
  8. 2019Shift R-CNN: Deep Monocular 3D Object Detection with Closed-Form Geometric Constraintscited by 25
  9. 2019Learning Independent Object Motion from Unlabelled Stereoscopic Videoscited by 17
  10. 2013Estimating Uncertain Spatial Relationships in Roboticscited by 14

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 0; 2023: 0; 2024: 1; 2025: 1.

3 papers.

  1. 2025DynaGSLAM: Real-Time Gaussian-Splatting SLAM for Online Rendering, Tracking, Motion Predictions of Moving Objects in Dynamic Scenescited by 10
  2. 2024SplatSDF: Boosting SDF-NeRF via Architecture-Level Fusion with Gaussian Splatscited by 5
  3. 2021SoK: Vehicle Orientation Representations for Deep Rotation Estimationcited by 1