# Grounded-Exo2Ego: Structured Semantic Grounding for Robust Exocentric-to-Egocentric Video Generation

arXiv: https://arxiv.org/abs/2608.20534 · HTML: https://mlsanity.com/paper/2608.20534/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 32 references that are in the map.
- First posted on arXiv 20 Aug 2026.

## Builds on (most cited first, 27 of 27)

- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [SAM 2: Segment Anything in Images and Videos](https://mlsanity.com/paper/2408.00714/) (2024)
- [Masked-attention Mask Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2112.01527/) (2021)
- [VBench: Comprehensive Benchmark Suite for Video Generative Models](https://mlsanity.com/paper/2311.17982/) (2023)
- [Towards Accurate Generative Models of Video: A New Metric & Challenges](https://mlsanity.com/paper/1812.01717/) (2018)
- [SAM 3: Segment Anything with Concepts](https://mlsanity.com/paper/2511.16719/) (2025)
- [Depth Anything 3: Recovering the Visual Space from Any Views](https://mlsanity.com/paper/2511.10647/) (2025)
- [Modular Primitives for High-Performance Differentiable Rendering](https://mlsanity.com/paper/2011.03277/) (2020)
- [GEN3C: 3D-Informed World-Consistent Video Generation with Precise Camera Control](https://mlsanity.com/paper/2503.03751/) (2025)
- [π3: Permutation-Equivariant Visual Geometry Learning](https://mlsanity.com/paper/2507.13347/) (2025)
- [ReCamMaster: Camera-Controlled Generative Rendering from A Single Video](https://mlsanity.com/paper/2503.11647/) (2025)
- [MoGe-2: Accurate Monocular Geometry with Metric Scale and Sharp Details](https://mlsanity.com/paper/2507.02546/) (2025)
- [LTX-2: Efficient Joint Audio-Visual Foundation Model](https://mlsanity.com/paper/2601.03233/) (2026)
- [In-Context LoRA for Diffusion Transformers](https://mlsanity.com/paper/2410.23775/) (2024)
- [Infinite Photorealistic Worlds using Procedural Generation](https://mlsanity.com/paper/2306.09310/) (2023)
- [TrajectoryCrafter: Redirecting Camera Trajectory for Monocular Videos via Diffusion Models](https://mlsanity.com/paper/2503.05638/) (2025)
- [Cameras as Relative Positional Encoding](https://mlsanity.com/paper/2507.10496/) (2025)
- [SynCamMaster: Synchronizing Multi-Camera Video Generation from Diverse Viewpoints](https://mlsanity.com/paper/2412.07760/) (2024)
- [Uni3C: Unifying Precisely 3D-Enhanced Camera and Human Motion Controls for Video Generation](https://mlsanity.com/paper/2504.14899/) (2025)
- [CapRL: Stimulating Dense Image Caption Capabilities via Reinforcement Learning](https://mlsanity.com/paper/2509.22647/) (2025)
- [Unified Camera Positional Encoding for Controlled Video Generation](https://mlsanity.com/paper/2512.07237/) (2025)
- [Vista4D: Video Reshooting with 4D Point Clouds](https://mlsanity.com/paper/2604.21915/) (2026)
- [Exo2EgoSyn: Unlocking Foundation Video Generation Models for Exocentric-to-Egocentric Video Synthesis](https://mlsanity.com/paper/2511.20186/) (2025)
- [EgoWorld: Translating Exocentric View to Egocentric View using Rich Exocentric Observations](https://mlsanity.com/paper/2506.17896/) (2025)
- [Plenoptic Video Generation](https://mlsanity.com/paper/2601.05239/) (2026)
- [EgoX: Egocentric Video Generation from a Single Exocentric Video](https://mlsanity.com/paper/2512.08269/) (2025)

## Built on by (most cited first, 2 of 2)

- [Ego-Forge: Text and Geometric-Attention Free Exo-to-Egocentric Video Generation](https://mlsanity.com/paper/2609.35368/) (2026)
- [Exo2EgoHOI: Hand-Object-Interaction Aware Exocentric-to-Egocentric Video Generation](https://mlsanity.com/paper/2609.38615/) (2026)

## Abstract (arXiv)

Generating egocentric video from a single exocentric video is an emerging and important topic for AR/VR and physical AI. Compared with conventional novel view synthesis, exo-to-ego generation is a significantly harder task because the standard geometric conditioning becomes highly unreliable under extreme view changes and large unobservable regions. We present Grounded-Exo2Ego, a principled framework that addresses these challenges at both the architectural and data levels. Architecturally, Grounded-Exo2Ego is a dual-branch video diffusion model that couples a geometric anchoring branch, which conditions the generation on the rendering of a 3D reconstruction, with a novel semantic grounding branch, which goes beyond the prevailing geometry-based approach and improves quality by synthesizing challenging regions based on object-level context. Additionally, we found that the overlooked issue of camera-reconstruction misalignment severely undermines exo-to-ego learning. We thus introduce a camera re-localization algorithm that resolves this issue and substantially improves quality across all metrics. We further develop a fully automated synthetic data engine that generates and renders rigged 3D characters in procedurally generated environments. Evaluation on the challenging EgoExo4D dataset shows that our method outperforms recent state-of-the-art approaches by large margins across all metrics. Detailed ablations validate improvements from each of our contributions at both the data and architectural level.