Multi-View Fusion for Multi-Level Robotic Scene Understanding
Not on the map. Why?
Abstract (arXiv)
We present a system for multi-level scene awareness for robotic manipulation. Given a sequence of camera-in-hand RGB images, the system calculates three types of information: 1) a point cloud representation of all the surfaces in the scene, for the purpose of obstacle avoidance; 2) the rough pose of unknown objects from categories corresponding to primitive shapes (e.g., cuboids and cylinders); and 3) full 6-DoF pose of known objects. By developing and fusing recent techniques in these domains, we provide a rich scene representation for robot awareness. We demonstrate the importance of each of these modules, their complementary nature, and the potential benefits of the system in the context of robotic manipulation.
Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). First posted on arXiv 25 Mar 2021.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 1; 2022: 3; 2023: 3; 2024: 5; 2025: 3; 2026: 2.
17 papers.
Sort:
By area:
- 20226-DoF Pose Estimation of Household Objects for Robotic Manipulation: An Accessible Dataset and Benchmarkcited by 85
- 2022Multi-Robot Collaborative Perception with Graph Neural Networkscited by 34
- 2024DiffusionNOCS: Managing Symmetry and Uncertainty in Sim2Real Multi-Modal Category-level Pose Estimationcited by 24
- 2026RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policiescited by 19
- 2021From SLAM to Situational Awareness: Challenges and Surveycited by 14
- 2022RGB-Only Reconstruction of Tabletop Scenes for Collision-Free Manipulator Controlcited by 10
- 20256D Object Pose Tracking in Internet Videos for Robotic Manipulationcited by 10
- 2024Zero-Shot Multi-Object Scene Completioncited by 9
- 2023RIC: Rotate-Inpaint-Complete for Generalizable Scene Reconstructioncited by 3
- 2024Learning Embeddings with Centroid Triplet Loss for Object Identification in Robotic Graspingcited by 3
- 2025PhysPose: Refining 6D Object Poses with Physical Constraintscited by 3
- 2026IndustryShapes: An RGB-D Benchmark dataset for 6D object pose estimation of industrial assembly components and toolscited by 1
5 more
- 20233 Dimensional Dense Reconstruction: A Review of Algorithms and Datasetcited by 0
- 2023Object Pose Estimation Annotation Pipeline for Multi-view Monocular Camera Systems in Industrial Settingscited by 0
- 2024RenDetNet: Weakly-supervised Shadow Detection with Shadow Caster Verificationcited by 0
- 2024GUMBEL-NERF: Representing Unseen Objects as Part-Compositional Neural Radiance Fieldscited by 0
- 2025Bridging the Ex-Vivo to In-Vivo Gap: Synthetic Priors for Monocular Depth Estimation in Specular Surgical Environmentscited by 0
- 2026RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policies10 Apr 2026
- 2026IndustryShapes: An RGB-D Benchmark dataset for 6D object pose estimation of industrial assembly components and tools5 Feb 2026
- 2025Bridging the Ex-Vivo to In-Vivo Gap: Synthetic Priors for Monocular Depth Estimation in Specular Surgical Environments29 Dec 2025
- 2025PhysPose: Refining 6D Object Poses with Physical Constraints30 Mar 2025
- 20256D Object Pose Tracking in Internet Videos for Robotic Manipulation13 Mar 2025
- 2024GUMBEL-NERF: Representing Unseen Objects as Part-Compositional Neural Radiance Fields27 Oct 2024
- 2024RenDetNet: Weakly-supervised Shadow Detection with Shadow Caster Verification30 Aug 2024
- 2024Learning Embeddings with Centroid Triplet Loss for Object Identification in Robotic Grasping9 Apr 2024
- 2024Zero-Shot Multi-Object Scene Completion21 Mar 2024
- 2024DiffusionNOCS: Managing Symmetry and Uncertainty in Sim2Real Multi-Modal Category-level Pose Estimation20 Feb 2024
- 2023Object Pose Estimation Annotation Pipeline for Multi-view Monocular Camera Systems in Industrial Settings23 Oct 2023
- 2023RIC: Rotate-Inpaint-Complete for Generalizable Scene Reconstruction21 Jul 2023
5 more
- 20233 Dimensional Dense Reconstruction: A Review of Algorithms and Dataset19 Apr 2023
- 2022RGB-Only Reconstruction of Tabletop Scenes for Collision-Free Manipulator Control21 Oct 2022
- 20226-DoF Pose Estimation of Household Objects for Robotic Manipulation: An Accessible Dataset and Benchmark11 Mar 2022
- 2022Multi-Robot Collaborative Perception with Graph Neural Networks5 Jan 2022
- 2021From SLAM to Situational Awareness: Challenges and Survey1 Oct 2021
- 20226-DoF Pose Estimation of Household Objects for Robotic Manipulation: An Accessible Dataset and Benchmarkcited by 85
- 2022Multi-Robot Collaborative Perception with Graph Neural Networkscited by 34
- 2024DiffusionNOCS: Managing Symmetry and Uncertainty in Sim2Real Multi-Modal Category-level Pose Estimationcited by 24
- 2026RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policiescited by 19
- 2021From SLAM to Situational Awareness: Challenges and Surveycited by 14
- 2022RGB-Only Reconstruction of Tabletop Scenes for Collision-Free Manipulator Controlcited by 10
- 20256D Object Pose Tracking in Internet Videos for Robotic Manipulationcited by 10
- 2024Zero-Shot Multi-Object Scene Completioncited by 9
- 2023RIC: Rotate-Inpaint-Complete for Generalizable Scene Reconstructioncited by 3
- 2024Learning Embeddings with Centroid Triplet Loss for Object Identification in Robotic Graspingcited by 3
- 2025PhysPose: Refining 6D Object Poses with Physical Constraintscited by 3
- 2026IndustryShapes: An RGB-D Benchmark dataset for 6D object pose estimation of industrial assembly components and toolscited by 1
12 most cited of 16 All 16 (JSON)
- 2026RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policies10 Apr 2026
- 2026IndustryShapes: An RGB-D Benchmark dataset for 6D object pose estimation of industrial assembly components and tools5 Feb 2026
- 2025Bridging the Ex-Vivo to In-Vivo Gap: Synthetic Priors for Monocular Depth Estimation in Specular Surgical Environments29 Dec 2025
- 2025PhysPose: Refining 6D Object Poses with Physical Constraints30 Mar 2025
- 20256D Object Pose Tracking in Internet Videos for Robotic Manipulation13 Mar 2025
- 2024RenDetNet: Weakly-supervised Shadow Detection with Shadow Caster Verification30 Aug 2024
- 2024Learning Embeddings with Centroid Triplet Loss for Object Identification in Robotic Grasping9 Apr 2024
- 2024Zero-Shot Multi-Object Scene Completion21 Mar 2024
- 2024DiffusionNOCS: Managing Symmetry and Uncertainty in Sim2Real Multi-Modal Category-level Pose Estimation20 Feb 2024
- 2023Object Pose Estimation Annotation Pipeline for Multi-view Monocular Camera Systems in Industrial Settings23 Oct 2023
- 2023RIC: Rotate-Inpaint-Complete for Generalizable Scene Reconstruction21 Jul 2023
- 20233 Dimensional Dense Reconstruction: A Review of Algorithms and Dataset19 Apr 2023
12 newest of 16 All 16 (JSON)