mlsanityData to 14 Oct 2026

Reconstruction by Generation: 3D Multi-Object Scene Reconstruction from Sparse Observations

arXiv 2604.27106 ↗First posted 29 Apr 2026cs.CV cs.AI cs.LGcited by 7

Not on the map. Why?

Abstract (arXiv)

Accurately reconstructing complex full multi-object scenes from sparse observations remains a core challenge in computer vision and a key step toward scalable and reliable simulation for robotics. In this work, we introduce RecGen, a generative framework for probabilistic joint estimation of object and part shapes, as well as their pose under occlusion and partial visibility from one or multiple RGB-D images. By leveraging compositional synthetic scene generation and strong 3D shape priors, RecGen generalizes across diverse object types and real-world environments. RecGen achieves state-of-the-art performance on complex, heavily occluded datasets, robustly handling severe occlusions, symmetric objects, object parts, and intricate geometry and texture. Despite using nearly 80% fewer training meshes than the previous state of the art SAM3D, RecGen outperforms it by 30.1% in geometric shape quality, 9.1% in texture reconstruction, and 33.9% in pose estimation.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 78 of its 88 references that are in the map. First posted on arXiv 29 Apr 2026.

Builds on

78 of 88 references.

References · 78

  1. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  2. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
  3. 2022Scalable Diffusion Models with Transformerscited by 6,054
  4. 2022Flow Matching for Generative Modelingcited by 5,702
  5. 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,489
  6. 2018On the Continuity of Rotation Representations in Neural Networkscited by 1,214
  7. 2019Habitat: A Platform for Embodied AI Researchcited by 1,191
  8. 2023Objaverse-XL: A Universe of 10M+ 3D Objectscited by 774
  9. 2024Structured 3D Latents for Scalable and Versatile 3D Generationcited by 770
  10. 2024LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creationcited by 739
  11. 2020SAPIEN: A SimulAted Part-based Interactive ENvironmentcited by 595
  12. 2025RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulationcited by 518
38 more
  1. 2024InstantMesh: Efficient 3D Mesh Generation from a Single Image with Sparse-view Large Reconstruction Modelscited by 433
  2. 2023FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objectscited by 429
  3. 2021ABO: Dataset and Benchmarks for Real-World 3D Object Understandingcited by 366
  4. 2025Hunyuan3D 2.0: Scaling Diffusion Models for High Resolution Textured 3D Assets Generationcited by 356
  5. 2018BOP: Benchmark for 6D Object Pose Estimationcited by 254
  6. 2025SAM 3D: 3Dfy Anything in Imagescited by 247
  7. 2024CRM: Single Image to 3D Textured Mesh with Convolutional Reconstruction Modelcited by 215
  8. 2025Native and Compact Structured Latents for 3D Generationcited by 175
  9. 2025FoundationStereo: Zero-Shot Stereo Matchingcited by 173
  10. 2023Habitat Synthetic Scenes Dataset (HSSD-200): An Analysis of 3D Scene Scale and Realism Tradeoffs for ObjectGoal Navigationcited by 167
  11. 2024Reconciling Reality through Simulation: A Real-to-Sim-to-Real Approach for Robust Manipulationcited by 144
  12. 2023Flexible Isosurface Extraction for Gradient-Based Mesh Optimizationcited by 139
  13. 2023Distilled Feature Fields Enable Few-Shot Language-Guided Manipulationcited by 130
  14. 2022Gen6D: Generalizable Model-Free 6-DoF Object Pose Estimation from RGB Imagescited by 121
  15. 2023PARIS: Part-level Reconstruction and Motion Analysis for Articulated Objectscited by 118
  16. 2024MIDI: Multi-Instance Diffusion for Single Image to 3D Scene Generationcited by 117
  17. 2023GigaPose: Fast and Robust Novel Object Pose Estimation via One Correspondencecited by 114
  18. 2019HomebrewedDB: RGB-D Dataset for 6D Pose Estimation of 3D Objectscited by 110
  19. 2024Articulate-Anything: Automatic Modeling of Articulated Objects via a Vision-Language Foundation Modelcited by 96
  20. 20226-DoF Pose Estimation of Household Objects for Robotic Manipulation: An Accessible Dataset and Benchmarkcited by 86
  21. 2025PartCrafter: Structured 3D Mesh Generation via Compositional Latent Diffusion Transformerscited by 84
  22. 2024Hunyuan3D 1.0: A Unified Framework for Text-to-3D and Image-to-3D Generationcited by 82
  23. 2022CenterSnap: Single-Shot Multi-Object 3D Shape Reconstruction and Categorical 6D Pose and Size Estimationcited by 70
  24. 2024SplatSim: Zero-Shot Sim2Real Transfer of RGB Manipulation Policies Using Gaussian Splattingcited by 69
  25. 2024PartGen: Part-level 3D Generation and Reconstruction with Multi-View Diffusion Modelscited by 62
  26. 2024Gen3DSR: Generalizable 3D Scene Reconstruction via Divide and Conquer from a Single Viewcited by 61
  27. 2024Robot See Robot Do: Imitating Articulated Object Manipulation with Monocular 4D Reconstructioncited by 54
  28. 2025Any6D: Model-free 6D Pose Estimation of Novel Objectscited by 49
  29. 2025Novel Demonstration Generation with Gaussian Splatting Enables Robust One-Shot Manipulationcited by 48
  30. 2024Dream to Manipulate: Compositional World Models Empowering Robot Imitation Learning with Imaginationcited by 45
  31. 2025Real2Render2Real: Scaling Robot Data Without Dynamics Simulation or Robot Hardwarecited by 44
  32. 2025Re3Sim: Generating High-Fidelity Simulation Data via 3D-Photorealistic Real-to-Sim for Robotic Manipulationcited by 43
  33. 2024Learning with 3D rotations, a hitchhiker's guide to SO(3)cited by 42
  34. 2025PhysX-3D: Physical-Grounded 3D Asset Generationcited by 42
  35. 2024GraspSplats: Efficient Manipulation with 3D Feature Splattingcited by 41
  36. 2024Physically Embodied Gaussian Splatting: A Realtime Correctable World Model for Roboticscited by 40
  37. 2024Splat-MOVER: Multi-Stage, Open-Vocabulary Robotic Manipulation via Editable Gaussian Splattingcited by 38
  38. 2025ArticulatedGS: Self-supervised Digital Twin Modeling of Articulated Objects using 3D Gaussian Splattingcited by 35

50 most cited of 78 All 78 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 7.

7 papers.

  1. 2026Lucida: Parse, Generate, and Place for Composable Real-to-Sim Scene Modelingcited by 2
  2. 2026SceneReGen: Generative Reconstruction of 3D Scenes from a Single Imagecited by 0
  3. 2026CODA: Depth-Aligned Scene Completion and Object Decomposition from a Single RGB-D Imagecited by 0
  4. 2026LEGO-Anything: Coding Agents for 3D Scene Reconstructioncited by 0
  5. 2026DecomVoxel: Harnessing 3D-Native Priors with Guided In-situ Denoising Optimization for Decompositional Scene Reconstructioncited by 0
  6. 2026Building Rome from a Single Imagecited by 0
  7. 2026GenIA: Generative Reconstruction with Test-Time Input Alignmentcited by 0