mlsanityData to 14 Oct 2026

Ego-Forge: Text and Geometric-Attention Free Exo-to-Egocentric Video Generation

arXiv 2609.35368 ↗First posted 28 Sep 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Exo-to-egocentric video generation aims to synthesize what a person sees from their own viewpoint given third-person footage and a target head trajectory. The task requires transferring appearance and semantics across large viewpoint changes while hallucinating content never observed by the exocentric camera. Existing approaches either impose additional input requirements, such as a ground-truth initial egocentric frame or multiple synchronized exocentric views, or remain limited to category-specific settings. EgoX is the first to address cross-activity and in-the-wild generalization, but requires a human-provided caption of the non-existent egocentric view at inference and introduces a computationally expensive geometry-guided attention bias that can propagate reconstruction errors and suppress textual and visual context. We therefore propose Ego-Forge, a caption-free and bias-free framework for exo-to-egocentric generation. It introduces textitDynamic Captioning, which derives conditioning tokens directly from the model's hidden states and adapts them to the diffusion timestep and network depth, replacing external text conditioning. By scaling training by an order of magnitude and using all available exocentric viewpoints, Ego-Forge learns cross-view correspondence implicitly and eliminates the need for geometry-guided attention, requiring only a lightweight depth prior. Ego-Forge achieves state-of-the-art performance on Ego-Exo4D, runs faster end-to-end, requires no external annotation at inference, and generalizes to in-the-wild scenes, including cases where over-reliance on geometry blocks appearance inference. Our model and source code will be made publicly available.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 45 of its 54 references that are in the map. First posted on arXiv 28 Sep 2026.

Builds on

45 of 54 references.

References · 45

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  3. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  4. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  5. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
  6. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
  7. 2025Qwen3-VL Technical Reportcited by 2,793
  8. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
  9. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,261
  10. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,556
  11. 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Modelscited by 1,464
  12. 2022R3M: A Universal Visual Representation for Robot Manipulationcited by 694
33 more
  1. 2023MotionCtrl: A Unified and Flexible Motion Controller for Video Generationcited by 580
  2. 2023Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectivescited by 490
  3. 2025VACE: All-in-One Video Creation and Editingcited by 479
  4. 2024CameraCtrl: Enabling Camera Control for Text-to-Video Generationcited by 460
  5. 2022VIP: Towards Universal Visual Reward and Representation via Value-Implicit Pre-Trainingcited by 395
  6. 2024OpenVid-1M: A Large-Scale High-Quality Dataset for Text-to-video Generationcited by 341
  7. 2025GEN3C: 3D-Informed World-Consistent Video Generation with Precise Camera Controlcited by 341
  8. 2023Show-1: Marrying Pixel and Latent Diffusion Models for Text-to-Video Generationcited by 302
  9. 2025ReCamMaster: Camera-Controlled Generative Rendering from A Single Videocited by 287
  10. 2025MoGe-2: Accurate Monocular Geometry with Metric Scale and Sharp Detailscited by 261
  11. 2023Project Aria: A New Tool for Egocentric Multi-Modal AI Researchcited by 208
  12. 2025History-Guided Video Diffusioncited by 171
  13. 2025ViPE: Video Pose Engine for 3D Geometric Perceptioncited by 170
  14. 2024Direct-a-Video: Customized Video Generation with User-Directed Camera Movement and Object Motioncited by 167
  15. 2025TrajectoryCrafter: Redirecting Camera Trajectory for Monocular Videos via Diffusion Modelscited by 125
  16. 2025VMem: Consistent Interactive Video Scene Generation with Surfel-Indexed View Memorycited by 109
  17. 2024EgoExoLearn: A Dataset for Bridging Asynchronous Ego- and Exo-centric View of Procedural Activities in Real Worldcited by 95
  18. 2024Evaluation Agent: Efficient and Promptable Evaluation Framework for Visual Generative Modelscited by 41
  19. 2024Put Myself in Your Shoes: Lifting the Egocentric Perspective from Exocentric Videoscited by 32
  20. 2024ObjectRelator: Enabling Cross-View Object Relation Understanding Across Ego-Centric and Exo-Centric Perspectivescited by 29
  21. 2023Learning Disentangled Identifiers for Action-Customized Text-to-Image Generationcited by 24
  22. 2025EgoExoBench: A Benchmark for First- and Third-person View Video Understanding in MLLMscited by 23
  23. 2025EgoExo-Gen: Ego-centric Video Prediction by Watching Exo-centric Videoscited by 19
  24. 2025Exo2EgoSyn: Unlocking Foundation Video Generation Models for Exocentric-to-Egocentric Video Synthesiscited by 15
  25. 2025EarthSynth: Generating Informative Earth Observation with Diffusion Modelscited by 12
  26. 2025EgoWorld: Translating Exocentric View to Egocentric View using Rich Exocentric Observationscited by 9
  27. 2025EgoX: Egocentric Video Generation from a Single Exocentric Videocited by 8
  28. 2025Control Copy-Paste: Controllable Diffusion-Based Augmentation Method for Remote Sensing Few-Shot Object Detectioncited by 7
  29. 2025Towards Comprehensive Scene Understanding: Integrating First and Third-Person Views for LVLMscited by 6
  30. 2025Diverse Instance Generation via Diffusion Models for Enhanced Few-Shot Object Detection in Remote Sensing Imagescited by 5
  31. 2026EgoExo-WM: Unlocking Exo Video for Ego World Modelscited by 3
  32. 2026Grounded-Exo2Ego: Structured Semantic Grounding for Robust Exocentric-to-Egocentric Video Generationcited by 2
  33. 2026From Synchrony to Sequence: Exo-to-Ego Generation via Interpolationcited by 1

Built on by

Not cited yet.