# Rendering Multi-Human and Multi-Object with 3D Gaussian Splatting

arXiv: https://arxiv.org/abs/2604.02996 · HTML: https://mlsanity.com/paper/2604.02996/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 39 of its 54 references that are in the map.
- First posted on arXiv 3 Apr 2026.

## Builds on (most cited first, 30 of 39)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [D-NeRF: Neural Radiance Fields for Dynamic Scenes](https://mlsanity.com/paper/2011.13961/) (2020)
- [Nerfies: Deformable Neural Radiance Fields](https://mlsanity.com/paper/2011.12948/) (2020)
- [HumanNeRF: Free-viewpoint Rendering of Moving People from Monocular Video](https://mlsanity.com/paper/2201.04127/) (2022)
- [Space-time Neural Irradiance Fields for Free-Viewpoint Video](https://mlsanity.com/paper/2011.12950/) (2020)
- [Animatable Neural Radiance Fields for Modeling Dynamic Human Bodies](https://mlsanity.com/paper/2105.02872/) (2021)
- [FSGS: Real-Time Few-shot View Synthesis using Gaussian Splatting](https://mlsanity.com/paper/2312.00451/) (2023)
- [ARCH: Animatable Reconstruction of Clothed Humans](https://mlsanity.com/paper/2004.04572/) (2020)
- [3DGS-Avatar: Animatable Avatars via Deformable 3D Gaussian Splatting](https://mlsanity.com/paper/2312.09228/) (2023)
- [CityGaussian: Real-time High-quality Large-Scale Scene Rendering with Gaussians](https://mlsanity.com/paper/2404.01133/) (2024)
- [CPF: Learning a Contact Potential Field to Model the Hand-Object Interaction](https://mlsanity.com/paper/2012.00924/) (2020)
- [CoR-GS: Sparse-View 3D Gaussian Splatting via Co-Regularization](https://mlsanity.com/paper/2405.12110/) (2024)
- [NeRF-DS: Neural Radiance Fields for Dynamic Specular Objects](https://mlsanity.com/paper/2303.14435/) (2023)
- [GauFRe: Gaussian Deformation Fields for Real-time Dynamic Novel View Synthesis](https://mlsanity.com/paper/2312.11458/) (2023)
- [HumanGaussian: Text-Driven 3D Human Generation with Gaussian Splatting](https://mlsanity.com/paper/2311.17061/) (2023)
- [SHERF: Generalizable Human NeRF from a Single Image](https://mlsanity.com/paper/2303.12791/) (2023)
- [HiFi4G: High-Fidelity Human Performance Rendering via Compact Gaussian Splatting](https://mlsanity.com/paper/2312.03461/) (2023)
- [NeuralDome: A Neural Modeling Pipeline on Multi-View Human-Object Interactions](https://mlsanity.com/paper/2212.07626/) (2022)
- [A Compact Dynamic 3D Gaussian Representation for Real-Time Dynamic View Synthesis](https://mlsanity.com/paper/2311.12897/) (2023)
- [Visibility Aware Human-Object Interaction Tracking from Single RGB Camera](https://mlsanity.com/paper/2303.16479/) (2023)
- [NeRF in the Palm of Your Hand: Corrective Augmentation for Robotics via Novel-View Synthesis](https://mlsanity.com/paper/2301.08556/) (2023)
- [MPS-NeRF: Generalizable 3D Human Rendering from Multiview Images](https://mlsanity.com/paper/2203.16875/) (2022)
- [HOI-M3:Capture Multiple Humans and Objects Interaction within Contextual Environment](https://mlsanity.com/paper/2404.00299/) (2024)
- [Learning Implicit Templates for Point-Based Clothed Human Modeling](https://mlsanity.com/paper/2207.06955/) (2022)
- [NeuralHOFusion: Neural Volumetric Rendering under Human-object Interactions](https://mlsanity.com/paper/2202.12825/) (2022)
- [CORE4D: A 4D Human-Object-Human Interaction Dataset for Collaborative Object REarrangement](https://mlsanity.com/paper/2406.19353/) (2024)
- [Neural Free-Viewpoint Performance Rendering under Complex Human-object Interactions](https://mlsanity.com/paper/2108.00362/) (2021)
- [HIMO: A New Benchmark for Full-Body Human Interacting with Multiple Objects](https://mlsanity.com/paper/2407.12371/) (2024)

## Abstract (arXiv)

Reconstructing dynamic scenes with multiple interacting humans and objects from sparse-view inputs is a critical yet challenging task, essential for creating high-fidelity digital twins for robotics and VR/AR. This problem, which we term Multi-Human Multi-Object (MHMO) rendering, presents two significant obstacles: achieving view-consistent representations for individual instances under severe mutual occlusion, and explicitly modeling the complex and combinatorial dependencies that arise from their interactions. To overcome these challenges, we propose MM-GS, a novel hierarchical framework built upon 3D Gaussian Splatting. Our method first employs a Per-Instance Multi-View Fusion module to establish a robust and consistent representation for each instance by aggregating visual information across all available views. Subsequently, a Scene-Level Instance Interaction module operates on a global scene graph to reason about relationships between all participants, refining their attributes to capture subtle interaction effects. Extensive experiments on challenging datasets demonstrate that our method significantly outperforms strong baselines, producing state-of-the-art results with high-fidelity details and plausible inter-instance contacts.