# Egocentric World Model for Photorealistic Hand-Object Interaction Synthesis

arXiv: https://arxiv.org/abs/2603.13615 · HTML: https://mlsanity.com/paper/2603.13615/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Placed in Embodied and robot world models, one of 11 directions in World models.
- Builds on 51 of its 53 references that are in the map.
- First posted on arXiv 13 Mar 2026.

## Placed in

- World models / [Embodied and robot world models](https://mlsanity.com/wm/embodied-and-robot-world-models/)

## Builds on (most cited first, 30 of 51)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Wan: Open and Advanced Large-Scale Video Generative Models](https://mlsanity.com/paper/2503.20314/) (2025)
- [Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets](https://mlsanity.com/paper/2311.15127/) (2023)
- [Video Diffusion Models](https://mlsanity.com/paper/2204.03458/) (2022)
- [DUSt3R: Geometric 3D Vision Made Easy](https://mlsanity.com/paper/2312.14132/) (2023)
- [Ego4D: Around the World in 3,000 Hours of Egocentric Video](https://mlsanity.com/paper/2110.07058/) (2021)
- [VGGT: Visual Geometry Grounded Transformer](https://mlsanity.com/paper/2503.11651/) (2025)
- [Cosmos World Foundation Model Platform for Physical AI](https://mlsanity.com/paper/2501.03575/) (2025)
- [VBench-2.0: Advancing Video Generation Benchmark Suite for Intrinsic Faithfulness](https://mlsanity.com/paper/2503.21755/) (2025)
- [MapAnything: Universal Feed-Forward Metric 3D Reconstruction](https://mlsanity.com/paper/2509.13414/) (2025)
- [Reconstructing Hands in 3D with Transformers](https://mlsanity.com/paper/2312.05251/) (2023)
- [CG-HOI: Contact-Guided 3D Human-Object Interaction Generation](https://mlsanity.com/paper/2311.16097/) (2023)
- [EgoVLA: Learning Vision-Language-Action Models from Egocentric Human Videos](https://mlsanity.com/paper/2507.12440/) (2025)
- [IMos: Intent-Driven Full-Body Motion Synthesis for Human-Object Interactions](https://mlsanity.com/paper/2212.07555/) (2022)
- [Controllable Human-Object Interaction Synthesis](https://mlsanity.com/paper/2312.03913/) (2023)
- [EgoScale: Scaling Dexterous Manipulation with Diverse Egocentric Human Data](https://mlsanity.com/paper/2602.16710/) (2026)
- [Tarsier2: Advancing Large Vision-Language Models from Detailed Video Description to Comprehensive Video Understanding](https://mlsanity.com/paper/2501.07888/) (2025)
- [GEM: A Generalizable Ego-Vision Multimodal World Model for Fine-Grained Ego-Motion, Object Dynamics, and Scene Composition Control](https://mlsanity.com/paper/2412.11198/) (2024)
- [Uni3C: Unifying Precisely 3D-Enhanced Camera and Human Motion Controls for Video Generation](https://mlsanity.com/paper/2504.14899/) (2025)
- [Being-H0.5: Scaling Human-Centric Robot Learning for Cross-Embodiment Generalization](https://mlsanity.com/paper/2601.12993/) (2026)
- [DiffH2O: Diffusion-Based Synthesis of Hand-Object Interactions from Textual Descriptions](https://mlsanity.com/paper/2403.17827/) (2024)
- [VLM4D: Towards Spatiotemporal Awareness in Vision Language Models](https://mlsanity.com/paper/2508.02095/) (2025)
- [Masquerade: Learning from In-the-wild Human Videos using Data-Editing](https://mlsanity.com/paper/2508.09976/) (2025)
- [Planning with Reasoning using Vision Language World Model](https://mlsanity.com/paper/2509.02722/) (2025)
- [GeneOH Diffusion: Towards Generalizable Hand-Object Interaction Denoising via Denoising Diffusion](https://mlsanity.com/paper/2402.14810/) (2024)
- [HOIGPT: Learning Long Sequence Hand-Object Interaction with Language Models](https://mlsanity.com/paper/2503.19157/) (2025)
- [Whole-Body Conditioned Egocentric Video Prediction](https://mlsanity.com/paper/2506.21552/) (2025)
- [InterHandGen: Two-Hand Interaction Generation via Cascaded Reverse Diffusion](https://mlsanity.com/paper/2403.17422/) (2024)

## Built on by (most cited first, 12 of 12)

- [AnchorWorld: Embodied Egocentric World Simulation with View-based Evolution Customization](https://mlsanity.com/paper/2606.07326/) (2026)
- [ZeroSplat: Generalized Referring Segmentation in 3D Gaussian Splatting](https://mlsanity.com/paper/2607.18801/) (2026)
- [HandsOnWorld: Unconstrained Egocentric Video Generation with Camera-Disentangled Hand Control](https://mlsanity.com/paper/2607.02075/) (2026)
- [EgoGenesis: Egocentric World-Action Modeling with Online Anchored Projective Memory and Action-3D RoPE](https://mlsanity.com/paper/2607.28243/) (2026)
- [EgoTac: In-the-wild Tactile Prediction from Egocentric Vision](https://mlsanity.com/paper/2608.15060/) (2026)
- [Hand-Object Interaction in the Age of Large Foundation Models:Reconstruction, Generation, and Embodied Transfer](https://mlsanity.com/paper/2607.28394/) (2026)
- [Human-Centric Intelligence in the Era of Foundation Models: A Survey](https://mlsanity.com/paper/2608.18184/) (2026)
- [Beyond Visual Quality: Evaluating Physical Consistency under Ego-Motion with EgoGenEval](https://mlsanity.com/paper/2609.11172/) (2026)
- [Code Plans, Diffusion Renders: Open-Ended Generative World Modeling](https://mlsanity.com/paper/2609.26458/) (2026)
- [DynamicHOI: Coupled Dynamics for Physics-aware HOI Reconstruction](https://mlsanity.com/paper/2609.36454/) (2026)
- [Exo2EgoHOI: Hand-Object-Interaction Aware Exocentric-to-Egocentric Video Generation](https://mlsanity.com/paper/2609.38615/) (2026)
- [EgoProbe: Benchmarking Long-Range and Exact Memory in Streaming Egocentric Videos](https://mlsanity.com/paper/2610.17128/) (2026)

## Abstract (arXiv)

To serve as a scalable data source for embodied AI, world models should act as true simulators that infer interaction dynamics strictly from user actions, rather than mere conditional video generators relying on privileged future object states. In this context, egocentric Human-Object Interaction (HOI) world models are critical for predicting physically grounded first-person rollouts. However, building such models is profoundly challenging due to rapid head motions, severe occlusions, and high-DoF hand articulations that abruptly alter contact topologies. Consequently, existing approaches often circumvent these physics challenges by resorting to conditional video generation with access to known future object trajectories. We introduce EgoHOI, an egocentric HOI world model that breaks away from this shortcut to simulate photorealistic, contact-consistent interactions from action signals alone. To ensure physical accuracy without future-state inputs, EgoHOI distills geometric and kinematic priors from 3D estimates into physics-informed embeddings. These embeddings regularize the egocentric rollouts toward physically valid dynamics. Experiments on the HOT3D dataset demonstrate consistent gains over strong baselines, and ablations validate the effectiveness of our physics-informed design.