Omni-Scan: Creating Visually-Accurate Digital Twin Object Models Using a Bimanual Robot with Handover and Gaussian Splat Merging
Not on the map. Why?
Abstract (arXiv)
3D Gaussian Splats (3DGSs) are 3D object models derived from multi-view images. Such "digital twins" are useful for simulations, virtual reality, marketing, robot policy fine-tuning, and part inspection. 3D object scanning usually requires multi-camera arrays, precise laser scanners, or robot wrist-mounted cameras, which have restricted workspaces. We propose Omni-Scan, a pipeline for producing high-quality 3D Gaussian Splat models using a bi-manual robot that grasps an object with one gripper and rotates the object with respect to a stationary camera. The object is then re-grasped by a second gripper to expose surfaces that were occluded by the first gripper. We present the Omni-Scan robot pipeline using DepthAny-thing, Segment Anything, as well as RAFT optical flow models to identify and isolate objects held by a robot gripper while removing the gripper and the background. We then modify the 3DGS training pipeline to support concatenated datasets with gripper occlusion, producing an omni-directional (360 degree view) model of the object. We apply Omni-Scan to part defect inspection, finding that it can identify visual or geometric defects in 12 different industrial and household objects with an average accuracy of 83%. Interactive videos of Omni-Scan 3DGS models can be found at https://berkeleyautomation.github.io/omni-scan/
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 40 of its 52 references that are in the map. First posted on arXiv 1 Aug 2025.
Builds on
40 of 52 references.
References · 40
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
- 2023Segment Anythingcited by 8,234
- 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
- 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,723
- 2024SAM 2: Segment Anything in Images and Videoscited by 2,863
- 2021Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fieldscited by 2,032
- 2021Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fieldscited by 1,964
- 2021Plenoxels: Radiance Fields without Neural Networkscited by 1,613
- 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,540
- 2020D-NeRF: Neural Radiance Fields for Dynamic Scenescited by 1,465
- 2024Depth Anything V2cited by 1,422
- 2022TensoRF: Tensorial Radiance Fieldscited by 1,280
28 more
- 2024Grounded SAM: Assembling Open-World Models for Diverse Visual Taskscited by 929
- 2023K-Planes: Explicit Radiance Fields in Space, Time, and Appearancecited by 746
- 2023Zip-NeRF: Anti-Aliased Grid-Based Neural Radiance Fieldscited by 671
- 2022Google Scanned Objects: A High-Quality Dataset of 3D Scanned Household Itemscited by 612
- 2023Nerfstudio: A Modular Framework for Neural Radiance Field Developmentcited by 600
- 2021NICE-SLAM: Neural Implicit Scalable Encoding for SLAMcited by 589
- 2021iMAP: Implicit Mapping and Positioning in Real-Timecited by 476
- 2024gsplat: An Open-Source Library for Gaussian Splattingcited by 285
- 2020TEASER: Fast and Certifiable Point Cloud Registrationcited by 274
- 2022DynIBaR: Neural Dynamic Image-Based Renderingcited by 224
- 2021Contact-GraspNet: Efficient 6-DoF Grasp Generation in Cluttered Scenescited by 218
- 2021HyperNeRF: A Higher-Dimensional Representation for Topologically Varying Neural Radiance Fieldscited by 214
- 2022NeRF-SLAM: Real-Time Dense Monocular SLAM with Neural Radiance Fieldscited by 188
- 2021Deblur-NeRF: Neural Radiance Fields from Blurry Imagescited by 179
- 2021Vision-Only Robot Navigation in a Neural Radiance Worldcited by 155
- 2023BundleSDF: Neural 6-DoF Tracking and 3D Reconstruction of Unknown Objectscited by 148
- 2023Distilled Feature Fields Enable Few-Shot Language-Guided Manipulationcited by 130
- 2024Grounding DINO 1.5: Advance the "Edge" of Open-Set Object Detectioncited by 122
- 2021Dex-NeRF: Using a Neural Radiance Field to Grasp Transparent Objectscited by 119
- 2023F2-NeRF: Fast Neural Radiance Field Training with Free Camera Trajectoriescited by 118
- 20213D Neural Scene Representations for Visuomotor Controlcited by 111
- 2023Language Embedded Radiance Fields for Zero-Shot Task-Oriented Graspingcited by 91
- 2024Robot See Robot Do: Imitating Articulated Object Manipulation with Monocular 4D Reconstructioncited by 54
- 2022NeRF2Real: Sim2real Transfer of Vision-guided Bipedal Motion Skills using Neural Radiance Fieldscited by 44
- 2022Reinforcement Learning with Neural Radiance Fieldscited by 43
- 2025Scalable Real2Sim: Physics-Aware Asset Generation Via Robotic Pick-and-Place Setupscited by 30
- 2021A Learned Stereo Depth System for Robotic Manipulation in Homescited by 14
- 2023Color-NeuS: Reconstructing Neural Implicit Surfaces with Colorcited by 7
Built on by
Citing papers per year: 2025: 1; 2026: 1.
2 papers.