mlsanityData to 7 Oct 2026

Hi4D: 4D Instance Segmentation of Close Human Interaction

arXiv 2303.15380 ↗First posted 27 Mar 2023cs.CVcited by 87

Not on the map. Why?

Abstract (arXiv)

We propose Hi4D, a method and dataset for the automatic analysis of physically close human-human interaction under prolonged contact. Robustly disentangling several in-contact subjects is a challenging task due to occlusions and complex shapes. Hence, existing multi-view systems typically fuse 3D surfaces of close subjects into a single, connected mesh. To address this issue we leverage i) individually fitted neural implicit avatars; ii) an alternating optimization scheme that refines pose and surface through periods of close proximity; and iii) thus segment the fused raw scans into individual instances. From these instances we compile Hi4D dataset of 4D textured scans of 20 subject pairs, 100 sequences, and a total of more than 11K frames. Hi4D contains rich interaction-centric annotations in 2D and 3D alongside accurately registered parametric body models. We define varied human pose and shape estimation tasks on this dataset and provide results from state-of-the-art methods on these benchmarks.

Key facts. Cited by 87 papers in the map (data through 7 Oct 2026). Builds on 67 of its 92 references that are in the map. First posted on arXiv 27 Mar 2023.

Builds on

67 of 92 references.

References · 67

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  3. 2017Mask R-CNNcited by 7,259
  4. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,829
  5. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  6. 2015Fast R-CNNcited by 2,930
  7. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
  8. 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,600
  9. 2019AMASS: Archive of Motion Capture as Surface Shapescited by 1,208
  10. 2016Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,061
  11. 2017End-to-end Recovery of Human Shape and Posecited by 883
  12. 2019PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitizationcited by 818
38 more
  1. 2016Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Imagecited by 692
  2. 2019Learning to Reconstruct 3D Human Pose and Shape via Model-fitting in the Loopcited by 625
  3. 2019VIBE: Video Inference for Human Body Pose and Shape Estimationcited by 615
  4. 2020PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitizationcited by 521
  5. 2019YOLACT: Real-time Instance Segmentationcited by 441
  6. 2019PointRend: Image Segmentation as Renderingcited by 439
  7. 2020GRAB: A Dataset of Whole-Body Human Grasping of Objectscited by 397
  8. 2021PARE: Part Attention Regressor for 3D Human Body Estimationcited by 334
  9. 2022ARCTIC: A Dataset for Dexterous Bimanual Hand-Object Manipulationcited by 306
  10. 2020HybrIK: A Hybrid Analytical-Neural Inverse Kinematics Solution for 3D Human Pose and Shape Estimationcited by 304
  11. 2019Resolving 3D Human Pose Ambiguities with 3D Scene Constraintscited by 294
  12. 2021ICON: Implicit Clothed humans Obtained from Normalscited by 283
  13. 2020ARCH: Animatable Reconstruction of Clothed Humanscited by 257
  14. 2022BEHAVE: Dataset and Method for Tracking Human Object Interactionscited by 246
  15. 2019Multi-Garment Net: Learning to Dress 3D People from Imagescited by 244
  16. 2018Total Capture: A 3D Deformation Model for Tracking Faces, Hands, and Bodiescited by 239
  17. 2018Video Based Reconstruction of 3D People Modelscited by 237
  18. 2020PaMIR: Parametric Model-Conditioned Implicit Representation for Image-based Human Reconstructioncited by 231
  19. 2021SNARF: Differentiable Forward Skinning for Animating Non-Rigid Neural Implicit Shapescited by 221
  20. 20183D-SIS: 3D Semantic Instance Segmentation of RGB-D Scanscited by 216
  21. 2020Monocular, One-stage, Regression of Multiple 3D Peoplecited by 216
  22. 2021AGORA: Avatars in Geography Optimized for Regression Analysiscited by 203
  23. 2017Single-Shot Multi-Person 3D Pose Estimation From Monocular RGBcited by 195
  24. 2021SCANimate: Weakly Supervised Learning of Skinned Clothed Avatar Networkscited by 181
  25. 2019Self-Correction for Human Parsingcited by 179
  26. 2022Capturing and Inferring Dense Full-Body Human-Scene Contactcited by 176
  27. 2019Learning to Reconstruct People in Clothing from a Single RGB Cameracited by 168
  28. 2020Monocular Expressive Body Regression through Body-Driven Attentioncited by 155
  29. 2019NASA: Neural Articulated Shape Approximationcited by 149
  30. 2022SelfRecon: Self Reconstruction Your Digital Avatar from Monocular Videocited by 144
  31. 2021Putting People in their Place: Monocular Regression of 3D People in Depthcited by 141
  32. 20203D-MPA: Multi Proposal Aggregation for 3D Semantic Instance Segmentationcited by 140
  33. 2021ContactOpt: Optimizing Contact to Improve Graspscited by 137
  34. 2020Coherent Reconstruction of Multiple Humans from a Single Imagecited by 136
  35. 2021Robust High-Resolution Video Matting with Temporal Guidancecited by 136
  36. 2023Vid2Avatar: 3D Avatar Reconstruction from Videos in the Wild via Self-supervised Scene Decompositioncited by 134
  37. 2020Combining Implicit Function Learning and Parametric Models for 3D Human Reconstructioncited by 133
  38. 2020Exemplar Fine-Tuning for 3D Human Model Fitting Towards In-the-Wild 3D Human Pose Estimationcited by 113

50 most cited of 67 All 67 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 6; 2024: 22; 2025: 34; 2026: 25.

87 papers.

Sort:

By area:

  1. 2024Sapiens: Foundation for Human Vision Modelscited by 229
  2. 2023Inter-X: Towards Versatile Human-Human Interaction Analysiscited by 98
  3. 2023HOLD: Category-agnostic 3D Reconstruction of Interacting Hands and Objects from Videocited by 76
  4. 2025PromptHMR: Promptable Human Mesh Recoverycited by 70
  5. 2025Go to Zero: Towards Zero-shot Motion Generation with Million-scale Datacited by 67
  6. 2026SAM 3D Body: Robust Full-Body Human Mesh Recoverycited by 61
  7. 2024Neural Localizer Fields for Continuous 3D Human Pose and Shape Estimationcited by 57
  8. 2023Generative Proxemics: A Prior for 3D Social Interaction from Imagescited by 43
  9. 2024Harmony4D: A Video Dataset for In-The-Wild Close Human Interactionscited by 31
  10. 2025Vid2Avatar-Pro: Authentic Avatar from Videos in the Wild via Universal Priorcited by 31
  11. 2025Ready-to-React: Online Reaction Policy for Two-Character Interaction Generationcited by 25
  12. 2025A Survey on Human Interaction Motion Generationcited by 25
38 more
  1. 2024Closely Interactive Human Reconstruction with Proxemics and Physics-Guided Adaptioncited by 18
  2. 2024TriDi: Trilateral Diffusion of 3D Humans, Objects, and Interactionscited by 18
  3. 2024PKU-DyMVHumans: A Multi-View Video Benchmark for High-Fidelity Dynamic Human Modelingcited by 17
  4. 2024Guess The Unseen: Dynamic 3D Scene Reconstruction from Partial 2D Glimpsescited by 17
  5. 2024Dense Hand-Object(HO) GraspNet with Full Grasping Taxonomy and Dynamicscited by 15
  6. 2025HERO: Human Reaction Generation from Videoscited by 15
  7. 2023Reconstructing Three-Dimensional Models of Interacting Humanscited by 14
  8. 2024MotionBank: A Large-scale Video Motion Benchmark with Disentangled Rule-based Annotationscited by 14
  9. 2025WorldPose: A World Cup Dataset for Global 3D Human Pose Estimationcited by 13
  10. 2025Reconstructing 4D Spatial Intelligence: A Surveycited by 13
  11. 20234DHumanOutfit: a multi-subject 4D dataset of human motion sequences in varying outfits exhibiting large displacementscited by 12
  12. 2024MultiPly: Reconstruction of Multiple People from Monocular Video in the Wildcited by 12
  13. 2025Learning Dense Hand Contact Estimation from Imbalanced Datacited by 12
  14. 2025DanceTogether! Identity-Preserving Multi-Person Interactive Video Generationcited by 12
  15. 2025ATLAS: Decoupling Skeletal and Shape Parameters for Expressive Parametric Human Modelingcited by 12
  16. 2026Video Generation Models are General-Purpose Vision Learnerscited by 12
  17. 2025DAViD: Data-efficient and Accurate Vision Models from Synthetic Datacited by 11
  18. 2025Human Mesh Modeling for Anny Bodycited by 10
  19. 2026LLaMo: Scaling Pretrained Language Models for Unified Motion Understanding and Generation with Continuous Autoregressive Tokenscited by 10
  20. 2025MVHumanNet++: A Large-scale Dataset of Multi-view Daily Dressing Human Captures with Richer Annotations for 3D Human Digitizationcited by 9
  21. 2025MHR: Momentum Human Rigcited by 9
  22. 2024Diffusion Models are Efficient Data Generators for Human Mesh Recoverycited by 8
  23. 2024Synergy and Synchrony in Couple Dancescited by 8
  24. 2023Ins-HOI: Instance Aware Human-Object Interactions Recoverycited by 7
  25. 2024InterAct: Capture and Modelling of Realistic, Expressive and Interactive Activities between Two Persons in Daily Scenarioscited by 7
  26. 2024AvatarPose: Avatar-guided 3D Pose Estimation of Close Human Interaction from Sparse Multi-view Videoscited by 7
  27. 2025Perceiving and Acting in First-Person: A Dataset and Benchmark for Egocentric Human-Object-Human Interactionscited by 7
  28. 2025Ponimator: Unfolding Interactive Pose for Versatile Human-human Interaction Animationcited by 7
  29. 2026From Frames to Sequences: Temporally Consistent Human-Centric Dense Predictioncited by 7
  30. 2024Pose Priors from Language Modelscited by 6
  31. 20253D Human Interaction Generation: A Surveycited by 6
  32. 2025ADHMR: Aligning Diffusion-based Human Mesh Recovery via Direct Preference Optimizationcited by 6
  33. 2025Interact2Ar: Full-Body Human-Human Interaction Generation via Autoregressive Diffusion Modelscited by 6
  34. 2024Multi-hypotheses Conditioned Point Cloud Diffusion for 3D Human Reconstruction from Occluded Imagescited by 5
  35. 2024CondiMen: Conditional Multi-Person Mesh Recoverycited by 5
  36. 2025Reconstructing Close Human Interaction with Appearance and Proxemics Reasoningcited by 4
  37. 2026THFM: A Unified Video Foundation Model for 4D Human Perception and Beyondcited by 4
  38. 2026Syn4D: A Multiview Synthetic 4D Datasetcited by 4

50 most cited of 87 All 87 (JSON)