mlsanityData to 7 Oct 2026

FunRec: Reconstructing Functional 3D Scenes from Egocentric Interaction Videos

arXiv 2604.05621 ↗First posted 7 Apr 2026cs.CVcited by 7

Not on the map. Why?

Abstract (arXiv)

We present FunRec, a method for reconstructing functional 3D digital twins of indoor scenes directly from egocentric RGB-D interaction videos. Unlike existing methods on articulated reconstruction, which rely on controlled setups, multi-state captures, or CAD priors, FunRec operates directly on in-the-wild human interaction sequences to recover interactable 3D scenes. It automatically discovers articulated parts, estimates their kinematic parameters, tracks their 3D motion, and reconstructs static and moving geometry in canonical space, yielding simulation-compatible meshes. Across new real and simulated benchmarks, FunRec surpasses prior work by a large margin, achieving up to +50 mIoU improvement in part segmentation, 5-10 times lower articulation and pose errors, and significantly higher reconstruction accuracy. We further demonstrate applications on URDF/USD export for simulation, hand-guided affordance mapping and robot-scene interaction.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 67 of its 83 references that are in the map. First posted on arXiv 7 Apr 2026.

Builds on

67 of 83 references.

References · 67

  1. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  2. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
  3. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  4. 2024SAM 2: Segment Anything in Images and Videoscited by 2,784
  5. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
  6. 2023ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenescited by 747
  7. 2025Continuous 3D Perception Model with Persistent Statecited by 527
  8. 2023CoTracker: It is Better to Track Togethercited by 516
  9. 2021ARKitScenes: A Diverse Real-World Dataset For 3D Indoor Scene Understanding Using Mobile RGB-D Datacited by 442
  10. 2024MonST3R: A Simple Approach for Estimating Geometry in the Presence of Motioncited by 431
  11. 2023FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objectscited by 411
  12. 2022HOI4D: A 4D Egocentric Dataset for Category-Level Human-Object Interactioncited by 321
38 more
  1. 2022ARCTIC: A Dataset for Dexterous Bimanual Hand-Object Manipulationcited by 306
  2. 2023Reconstructing Hands in 3D with Transformerscited by 298
  3. 2020PREDATOR: Registration of 3D Point Clouds with Low Overlapcited by 287
  4. 2021H2O: Two Hands Manipulating Objects for First Person Interaction Recognitioncited by 251
  5. 2024Shape of Motion: 4D Reconstruction from a Single Videocited by 247
  6. 2024MegaSaM: Accurate, Fast, and Robust Structure and Motion from Casual Dynamic Videoscited by 224
  7. 2023Tracking Everything Everywhere All at Oncecited by 190
  8. 2019RIO: 3D Object Instance Re-Localization in Changing Indoor Environmentscited by 187
  9. 2024SpatialTracker: Tracking Any 2D Pixels in 3D Spacecited by 178
  10. 2024MoSca: Dynamic Gaussian Fusion from Casual Videos via 4D Motion Scaffoldscited by 177
  11. 2022Ditto: Building Digital Twins of Articulated Objects from Interactioncited by 152
  12. 2020iGibson 1.0: a Simulation Environment for Interactive Tasks in Large Realistic Scenescited by 151
  13. 2023BundleSDF: Neural 6-DoF Tracking and 3D Reconstruction of Unknown Objectscited by 144
  14. 2024BEHAVIOR-1K: A Human-Centered, Embodied AI Benchmark with 1,000 Everyday Activities and Realistic Simulationcited by 140
  15. 2022EPIC-KITCHENS VISOR Benchmark: VIdeo Segmentations and Object Relationscited by 139
  16. 2024HOT3D: Hand and Object Tracking in 3D from Egocentric Multi-View Videoscited by 122
  17. 2024WiLoR: End-to-end 3D Hand Localization and Reconstruction in-the-wildcited by 117
  18. 2023PARIS: Part-level Reconstruction and Motion Analysis for Articulated Objectscited by 115
  19. 2023OneFormer3D: One Transformer for Unified Point Cloud Segmentationcited by 95
  20. 2024Articulate-Anything: Automatic Modeling of Articulated Objects via a Vision-Language Foundation Modelcited by 90
  21. 2024Automated Creation of Digital Cousins for Robust Policy Learningcited by 88
  22. 2025St4RTrack: Simultaneous 4D Reconstruction and Tracking in the Worldcited by 88
  23. 2021BundleTrack: 6D Pose Tracking for Novel Objects without Instance or Category-Level 3D Modelscited by 87
  24. 2024Neural Implicit Representation for Building Digital Twins of Unknown Articulated Objectscited by 67
  25. 2025ArtGS: Building Interactable Replicas of Complex Articulated Objects via Gaussian Splattingcited by 67
  26. 2025TAPIP3D: Tracking Any Point in Persistent 3D Geometrycited by 67
  27. 2023CARTO: Category and Joint Agnostic Reconstruction of ARTiculated Objectscited by 64
  28. 2024Real2Code: Reconstruct Articulated Objects via Code Generationcited by 64
  29. 2024DELTA: Dense Efficient Long-range 3D Tracking for any videocited by 62
  30. 2024ParaHome: Parameterizing Everyday Home Activities Towards 3D Generative Modeling of Human-Object Interactionscited by 54
  31. 2025AllTracker: Efficient Dense Point Tracking at High Resolutioncited by 54
  32. 2022OPD: Single-view 3D Openable Part Detectioncited by 52
  33. 2024Robot See Robot Do: Imitating Articulated Object Manipulation with Monocular 4D Reconstructioncited by 51
  34. 2024Cubify Anything: Scaling Indoor 3D Object Detectioncited by 39
  35. 2023OPDMulti: Openable Part Detection for Multiple Objectscited by 38
  36. 2025MetaScenes: Towards Automated Replica Creation for Real-world 3D Scanscited by 31
  37. 2025DRAWER: Digital Reconstruction and Articulation With Environment Realismcited by 29
  38. 2022Interaction Replica: Tracking Human-Object Interaction and Scene Changes From Human Motioncited by 27

50 most cited of 67 All 67 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 7.

7 papers.

  1. 2026EgoFun3D: Modeling Interactive Objects from Egocentric Videos using Function Templatescited by 1
  2. 2026Hierarchical and Holistic Open-Vocabulary Functional 3D Scene Graphs for Indoor Spacescited by 1
  3. 2026EgoPlay: Event-Triggered Video Editing for Egocentric Streamscited by 0
  4. 2026FunArt: Decoding Functional Structure and Articulation from Generative 3D Latentscited by 0
  5. 2026FAMOS: Feed-Forward 3D Articulation Modeling from Sparse Observationscited by 0
  6. 2026ForceTwin: Physics-informed Digital Twins for Robotic Manipulation from Instrumented Human Interactioncited by 0
  7. 2026AgentSTAR: Agentic Shape Tracking and Reconstruction from Monocular Videoscited by 0