mlsanityData to 7 Oct 2026

Localization and Mapping using Instance-specific Mesh Models

arXiv 2103.04493 ↗First posted 8 Mar 2021cs.CV cs.ROcited by 4

Not on the map. Why?

Abstract (arXiv)

This paper focuses on building semantic maps, containing object poses and shapes, using a monocular camera. This is an important problem because robots need rich understanding of geometry and context if they are to shape the future of transportation, construction, and agriculture. Our contribution is an instance-specific mesh model of object shape that can be optimized online based on semantic information extracted from camera images. Multi-view constraints on the object shape are obtained by detecting objects and extracting category-specific keypoints and segmentation masks. We show that the errors between projections of the mesh model and the observed keypoints and masks can be differentiated in order to obtain accurate instance-specific object shapes. We evaluate the performance of the proposed approach in simulation and on the KITTI dataset by building maps of car poses and shapes.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 19 of its 50 references that are in the map. First posted on arXiv 8 Mar 2021.

Builds on

19 of 50 references.

References · 19

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2017Mask R-CNNcited by 7,259
  3. 2013Rich feature hierarchies for accurate object detection and semantic segmentationcited by 2,160
  4. 2016A Point Set Generation Network for 3D Object Reconstruction from a Single Imagecited by 972
  5. 2016Stacked Hourglass Networks for Human Pose Estimationcited by 913
  6. 2016ORB-SLAM2: an Open-Source SLAM System for Monocular, Stereo and RGB-D Camerascited by 801
  7. 2017Neural 3D Mesh Renderercited by 437
  8. 2017VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimatorcited by 418
  9. 2016Past, Present, and Future of Simultaneous Localization And Mapping: Towards the Robust-Perception Agecited by 405
  10. 2018Learning Category-Specific Mesh Reconstruction from Image Collectionscited by 317
  11. 2017Multi-view Supervision for Single-view Reconstruction via Differentiable Ray Consistencycited by 177
  12. 2016Perspective Transformer Nets: Learning Single-View 3D Object Reconstruction without 3D Supervisioncited by 106
7 more
  1. 20176-DoF Object Pose from Semantic Keypointscited by 89
  2. 2018StarMap for Category-Agnostic Keypoint and Viewpoint Estimationcited by 42
  3. 2017CASENet: Deep Category-Aware Semantic Edge Detectioncited by 37
  4. 2018FutureMapping: The Computational Structure of Spatial AI Systemscited by 35
  5. 2017Robust Stereo Visual Inertial Odometry for Fast Autonomous Flightcited by 27
  6. 2018Stereo Vision-based Semantic 3D Object and Ego-motion Tracking for Autonomous Drivingcited by 26
  7. 2018Visual-Inertial Object Detection and Mappingcited by 5

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 2; 2023: 1.

4 papers.

  1. 2023MV-DeepSDF: Implicit Modeling with Multi-Sweep Point Clouds for 3D Vehicle Reconstruction in Autonomous Drivingcited by 11
  2. 2021Online Adaptation for Implicit Object Tracking and Shape Reconstruction in the Wildcited by 6
  3. 2022Large-Scale 3D Semantic Reconstruction for Automated Driving Vehicles with Adaptive Truncated Signed Distance Functioncited by 1
  4. 2022TerrainMesh: Metric-Semantic Terrain Reconstruction from Aerial Images Using Joint 2D-3D Learningcited by 0