mlsanityData to 14 Oct 2026

NeuralDome: A Neural Modeling Pipeline on Multi-View Human-Object Interactions

arXiv 2212.07626 ↗First posted 15 Dec 2022cs.CVcited by 68

Not on the map. Why?

Abstract (arXiv)

Humans constantly interact with objects in daily life tasks. Capturing such processes and subsequently conducting visual inferences from a fixed viewpoint suffers from occlusions, shape and texture ambiguities, motions, etc. To mitigate the problem, it is essential to build a training dataset that captures free-viewpoint interactions. We construct a dense multi-view dome to acquire a complex human object interaction dataset, named HODome, that consists of ~75M frames on 10 subjects interacting with 23 objects. To process the HODome dataset, we develop NeuralDome, a layer-wise neural processing pipeline tailored for multi-view video inputs to conduct accurate tracking, geometry reconstruction and free-view rendering, for both human subjects and objects. Extensive experiments on the HODome dataset demonstrate the effectiveness of NeuralDome on a variety of inference, modeling, and rendering tasks. Both the dataset and the NeuralDome tools will be disseminated to the community for further development.

Key facts. Cited by 68 papers in the map (data through 14 Oct 2026). Builds on 59 of its 87 references that are in the map. First posted on arXiv 15 Dec 2022.

Builds on

59 of 87 references.

References · 59

  1. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,723
  2. 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,712
  3. 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,612
  4. 2020D-NeRF: Neural Radiance Fields for Dynamic Scenescited by 1,465
  5. 2022TensoRF: Tensorial Radiance Fieldscited by 1,280
  6. 2016Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,064
  7. 2021PlenOctrees for Real-time Rendering of Neural Radiance Fieldscited by 905
  8. 2019PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitizationcited by 818
  9. 2021MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereocited by 740
  10. 2018Open3D: A Modern Library for 3D Data Processingcited by 721
  11. 2021IBRNet: Learning Multi-View Image-Based Renderingcited by 703
  12. 2020Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humanscited by 634
38 more
  1. 2021RegNeRF: Regularizing Neural Radiance Fields for View Synthesis from Sparse Inputscited by 615
  2. 2021Ref-NeRF: Structured View-Dependent Appearance for Neural Radiance Fieldscited by 575
  3. 2019NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understandingcited by 533
  4. 2020PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitizationcited by 521
  5. 2020Non-Rigid Neural Radiance Fields: Reconstruction and Novel View Synthesis of a Dynamic Scene From Monocular Videocited by 471
  6. 2017Neural 3D Mesh Renderercited by 437
  7. 2020GRAB: A Dataset of Whole-Body Human Grasping of Objectscited by 405
  8. 2021Animatable Neural Radiance Fields for Modeling Dynamic Human Bodiescited by 361
  9. 2019Neural Point-Based Graphicscited by 327
  10. 2019Resolving 3D Human Pose Ambiguities with 3D Scene Constraintscited by 295
  11. 2021Mixture of Volumetric Primitives for Efficient Neural Renderingcited by 277
  12. 2020ARCH: Animatable Reconstruction of Clothed Humanscited by 257
  13. 2022BEHAVE: Dataset and Method for Tracking Human Object Interactionscited by 249
  14. 2021HyperNeRF: A Higher-Dimensional Representation for Topologically Varying Neural Radiance Fieldscited by 214
  15. 2017Unite the People: Closing the Loop Between 3D and 2D Human Representationscited by 205
  16. 2021Neural Articulated Radiance Fieldcited by 196
  17. 2021Neural Rays for Occlusion-aware Image-based Renderingcited by 192
  18. 2022SparseNeuS: Fast Generalizable Neural Surface Reconstruction from Sparse Viewscited by 177
  19. 2021ARCH++: Animation-Ready Clothed Human Reconstruction Revisitedcited by 174
  20. 2016Panoptic Studio: A Massively Multiview System for Social Interaction Capturecited by 173
  21. 2019Neural Volumes: Learning Dynamic Renderable Volumes from Imagescited by 173
  22. 2022TAVA: Template-free Animatable Volumetric Actorscited by 172
  23. 2021FrankMocap: A Monocular 3D Whole-Body Pose Estimation System via Regression and Integrationcited by 169
  24. 2022Fourier PlenOctrees for Dynamic Radiance Field Rendering in Real-timecited by 165
  25. 2021Neural Human Performer: Learning Generalizable Radiance Fields for Human Performance Renderingcited by 158
  26. 2021GNeRF: GAN-based Neural Radiance Field without Posed Cameracited by 153
  27. 2020Perceiving 3D Human-Object Spatial Arrangements from a Single Image in the Wildcited by 152
  28. 2021Editable Free-viewpoint Video Using a Layered Neural Representationcited by 148
  29. 2022ARAH: Animatable Volume Rendering of Articulated Human SDFscited by 139
  30. 2021Real-time Deep Dynamic Characterscited by 133
  31. 2021HumanNeRF: Efficiently Generated Human Radiance Field from Sparse Inputscited by 112
  32. 2021Neural Actor: Neural Free-view Synthesis of Human Actors with Pose Controlcited by 106
  33. 2020Background Matting: The World is Your Green Screencited by 105
  34. 2022CHORE: Contact, Human and Object REconstruction from a single RGB imagecited by 95
  35. 2022InterCap: Joint Markerless 3D Tracking of Humans and Objects in Interactioncited by 93
  36. 2017Detailed, accurate, human shape estimation from clothed 3D scan sequencescited by 89
  37. 2019Textured Neural Avatarscited by 65
  38. 2022Human Performance Modeling and Rendering via Neural Animated Meshcited by 64

50 most cited of 59 All 59 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 10; 2024: 16; 2025: 23; 2026: 19.

68 papers.

Sort:

By area:

  1. 2023InterDiff: Generating 3D Human-Object Interactions with Physics-Informed Diffusioncited by 182
  2. 2023PhysHOI: Physics-Based Imitation of Dynamic Human-Object Interactioncited by 76
  3. 2023Diffusion-Guided Reconstruction of Everyday Hand-Object Interaction Clipscited by 74
  4. 2025InterMimic: Towards Universal Whole-Body Control for Physics-Based Human-Object Interactionscited by 72
  5. 2024InterDreamer: Zero-Shot Text to 3D Dynamic Human-Object Interactioncited by 64
  6. 2024ParaHome: Parameterizing Everyday Home Activities Towards 3D Generative Modeling of Human-Object Interactionscited by 57
  7. 2024HOI-M3:Capture Multiple Humans and Objects Interaction within Contextual Environmentcited by 53
  8. 2024THOR: Text to Human-Object Interaction Diffusion via Relation Interventioncited by 46
  9. 2024The Language of Motion: Unifying Verbal and Non-verbal Language of 3D Human Motioncited by 46
  10. 2023I'M HOI: Inertia-aware Monocular Capture of 3D Human-Object Interactionscited by 44
  11. 2024CORE4D: A 4D Human-Object-Human Interaction Dataset for Collaborative Object REarrangementcited by 44
  12. 2023Template Free Reconstruction of Human-object Interaction with Procedural Interaction Generationcited by 38
38 more
  1. 2024HIMO: A New Benchmark for Full-Body Human Interacting with Multiple Objectscited by 36
  2. 2023HOSNeRF: Dynamic Human-Object-Scene Neural Radiance Fields from a Single Videocited by 33
  3. 2025InterAct: Advancing Large-Scale Versatile 3D Human-Object Interaction Generationcited by 32
  4. 2025HUMOTO: A 4D Dataset of Mocap Human Object Interactionscited by 31
  5. 2024Recent Trends in 3D Reconstruction of General Non-Rigid Scenescited by 28
  6. 2025A Survey on Human Interaction Motion Generationcited by 25
  7. 2024InterTrack: Tracking Human Object Interaction without Object Templatescited by 23
  8. 2025The Quest for Generalizable Motion Generation: Data, Model, and Evaluationcited by 22
  9. 2024TriDi: Trilateral Diffusion of 3D Humans, Objects, and Interactionscited by 20
  10. 2024A Unified Diffusion Framework for Scene-aware Human Motion Estimation from Sparse Signalscited by 16
  11. 2024Mimicking-Bench: A Benchmark for Generalizable Humanoid-Scene Interaction Learning via Human Mimickingcited by 16
  12. 2025SemGeoMo: Dynamic Contextual Human Motion Generation with Semantic and Geometric Guidancecited by 16
  13. 2025CARI4D: Category Agnostic 4D Reconstruction of Human-Object Interactioncited by 14
  14. 2026InterPrior: Scaling Generative Control for Physics-Based Human-Object Interactionscited by 11
  15. 2024RHOBIN Challenge: Reconstruction of Human Object Interactioncited by 10
  16. 2025MVHumanNet++: A Large-scale Dataset of Multi-view Daily Dressing Human Captures with Richer Annotations for 3D Human Digitizationcited by 9
  17. 2025End-to-End HOI Reconstruction Transformer with Graph-based Encodingcited by 8
  18. 2023Replay: Multi-modal Multi-view Acted Videos for Casual Holographycited by 7
  19. 2023BOTH2Hands: Inferring 3D Hands from Both Text Prompts and Body Dynamicscited by 7
  20. 2023Ins-HOI: Instance Aware Human-Object Interactions Recoverycited by 7
  21. 20253D Human Interaction Generation: A Surveycited by 7
  22. 2025ViBES: A Conversational Agent with Behaviorally-Intelligent 3D Virtual Bodycited by 6
  23. 2026Unleashing Guidance Without Classifiers for Human-Object Interaction Animationcited by 6
  24. 2024Gaze-guided Hand-Object Interaction Synthesis: Dataset and Methodcited by 5
  25. 2024StackFLOW: Monocular Human-Object Reconstruction by Stacked Normalizing Flow with Offsetcited by 5
  26. 2025GASPACHO: Gaussian Splatting for Controllable Humans and Objectscited by 5
  27. 2025ECHO: Ego-Centric modeling of Human-Object interactionscited by 5
  28. 2025InterAgent: Physics-based Multi-agent Command Execution via Diffusion on Interaction Graphscited by 5
  29. 2025OmniMotion-X: Versatile Multimodal Whole-Body Motion Generationcited by 4
  30. 2025MMHOI: Modeling Complex 3D Multi-Human Multi-Object Interactionscited by 3
  31. 2025Efficient and Scalable Monocular Human-Object Interaction Motion Reconstructioncited by 3
  32. 2025Betsu-Betsu: Multi-View Separable 3D Reconstruction of Two Interacting Objectscited by 2
  33. 2025Learning to Generate Human-Human-Object Interactions from Textual Descriptionscited by 2
  34. 2026AnyLift: Scaling Motion Reconstruction from Internet Videos via 2D Diffusioncited by 2
  35. 2026Uni-HOI:A Unified framework for Learning the Joint distribution of Text and Human-Object Interactioncited by 2
  36. 2026HOIMask: Towards Generative Masked Modeling for Human Object Interaction Generationcited by 2
  37. 20243DArticCyclists: Generating Synthetic Articulated 8D Pose-Controllable Cyclist Data for Computer Vision Applicationscited by 1
  38. 2025Interaction-Aware 4D Gaussian Splatting for Dynamic Hand-Object Interaction Reconstructioncited by 1

50 most cited of 68 All 68 (JSON)