# Multi-View Fusion for Multi-Level Robotic Scene Understanding

arXiv: https://arxiv.org/abs/2103.13539 · HTML: https://mlsanity.com/paper/2103.13539/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Mar 2021.

## Built on by (most cited first, 17 of 17)

- [6-DoF Pose Estimation of Household Objects for Robotic Manipulation: An Accessible Dataset and Benchmark](https://mlsanity.com/paper/2203.05701/) (2022)
- [Multi-Robot Collaborative Perception with Graph Neural Networks](https://mlsanity.com/paper/2201.01760/) (2022)
- [DiffusionNOCS: Managing Symmetry and Uncertainty in Sim2Real Multi-Modal Category-level Pose Estimation](https://mlsanity.com/paper/2402.12647/) (2024)
- [RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policies](https://mlsanity.com/paper/2604.09860/) (2026)
- [From SLAM to Situational Awareness: Challenges and Survey](https://mlsanity.com/paper/2110.00273/) (2021)
- [RGB-Only Reconstruction of Tabletop Scenes for Collision-Free Manipulator Control](https://mlsanity.com/paper/2210.11668/) (2022)
- [6D Object Pose Tracking in Internet Videos for Robotic Manipulation](https://mlsanity.com/paper/2503.10307/) (2025)
- [Zero-Shot Multi-Object Scene Completion](https://mlsanity.com/paper/2403.14628/) (2024)
- [RIC: Rotate-Inpaint-Complete for Generalizable Scene Reconstruction](https://mlsanity.com/paper/2307.11932/) (2023)
- [Learning Embeddings with Centroid Triplet Loss for Object Identification in Robotic Grasping](https://mlsanity.com/paper/2404.06277/) (2024)
- [PhysPose: Refining 6D Object Poses with Physical Constraints](https://mlsanity.com/paper/2503.23587/) (2025)
- [IndustryShapes: An RGB-D Benchmark dataset for 6D object pose estimation of industrial assembly components and tools](https://mlsanity.com/paper/2602.05555/) (2026)
- [3 Dimensional Dense Reconstruction: A Review of Algorithms and Dataset](https://mlsanity.com/paper/2304.09371/) (2023)
- [Object Pose Estimation Annotation Pipeline for Multi-view Monocular Camera Systems in Industrial Settings](https://mlsanity.com/paper/2310.14914/) (2023)
- [RenDetNet: Weakly-supervised Shadow Detection with Shadow Caster Verification](https://mlsanity.com/paper/2408.17143/) (2024)
- [GUMBEL-NERF: Representing Unseen Objects as Part-Compositional Neural Radiance Fields](https://mlsanity.com/paper/2410.20306/) (2024)
- [Bridging the Ex-Vivo to In-Vivo Gap: Synthetic Priors for Monocular Depth Estimation in Specular Surgical Environments](https://mlsanity.com/paper/2512.23786/) (2025)

## Abstract (arXiv)

We present a system for multi-level scene awareness for robotic manipulation. Given a sequence of camera-in-hand RGB images, the system calculates three types of information: 1) a point cloud representation of all the surfaces in the scene, for the purpose of obstacle avoidance; 2) the rough pose of unknown objects from categories corresponding to primitive shapes (e.g., cuboids and cylinders); and 3) full 6-DoF pose of known objects. By developing and fusing recent techniques in these domains, we provide a rich scene representation for robot awareness. We demonstrate the importance of each of these modules, their complementary nature, and the potential benefits of the system in the context of robotic manipulation.