mlsanityData to 14 Oct 2026

DIV-FF: Dynamic Image-Video Feature Fields For Environment Understanding in Egocentric Videos

arXiv 2503.08344 ↗First posted 11 Mar 2025cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Environment understanding in egocentric videos is an important step for applications like robotics, augmented reality and assistive technologies. These videos are characterized by dynamic interactions and a strong dependence on the wearer engagement with the environment. Traditional approaches often focus on isolated clips or fail to integrate rich semantic and geometric information, limiting scene comprehension. We introduce Dynamic Image-Video Feature Fields (DIV FF), a framework that decomposes the egocentric scene into persistent, dynamic, and actor based components while integrating both image and video language features. Our model enables detailed segmentation, captures affordances, understands the surroundings and maintains consistent understanding over time. DIV-FF outperforms state-of-the-art methods, particularly in dynamically evolving scenarios, demonstrating its potential to advance long term, spatio temporal scene understanding.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 53 of its 60 references that are in the map. First posted on arXiv 11 Mar 2025.

Builds on

53 of 60 references.

References · 53

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2023Segment Anythingcited by 8,234
  3. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  4. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
  5. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,556
  6. 2022Reproducible scaling laws for contrastive language-image learningcited by 1,173
  7. 2018Scaling Egocentric Vision: The EPIC-KITCHENS Datasetcited by 795
  8. 2020Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenescited by 751
  9. 2023LERF: Language Embedded Radiance Fieldscited by 543
  10. 2023Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectivescited by 490
  11. 2018Gibson Env: Real-World Perception for Embodied Agentscited by 481
  12. 2021Dynamic View Synthesis from Dynamic Monocular Videocited by 420
38 more
  1. 2021In-Place Scene Labelling and Understanding with Implicit Scene Representationcited by 415
  2. 2023LangSplat: 3D Language Gaussian Splattingcited by 415
  3. 2022Decomposing NeRF for Editing via Feature Field Distillationcited by 371
  4. 2023Scaling Open-Vocabulary Object Detectioncited by 343
  5. 2020Understanding Human Hands in Contact at Internet Scalecited by 281
  6. 2023Project Aria: A New Tool for Egocentric Multi-Modal AI Researchcited by 208
  7. 2021Anticipative Video Transformercited by 200
  8. 2022Neural Feature Fusion Fields: 3D Distillation of Self-Supervised 2D Image Representationscited by 196
  9. 2024Hierarchical Open-Vocabulary 3D Scene Graphs for Language-Grounded Robot Navigationcited by 196
  10. 2022Panoptic Lifting for 3D Scene Understanding with Neural Fieldscited by 174
  11. 2022Panoptic NeRF: 3D-to-2D Label Transfer for Panoptic Urban Scene Segmentationcited by 152
  12. 2022D2NeRF: Self-Supervised Decoupling of Dynamic and Static Objects from a Monocular Videocited by 137
  13. 2023EgoVLPv2: Egocentric Video-Language Pre-training with Fusion in the Backbonecited by 130
  14. 2018Grounded Human-Object Interaction Hotspots from Videocited by 129
  15. 2021NeSF: Neural Semantic Fields for Generalizable Semantic Segmentation of 3D Scenescited by 119
  16. 2020EGO-TOPO: Environment Affordances from Egocentric Videocited by 111
  17. 2019What Would You Expect? Anticipating Egocentric Actions with Rolling-Unrolling LSTMs and Modality Attentioncited by 105
  18. 2024NeRF On-the-go: Exploiting Uncertainty for Distractor-free NeRFs in the Wildcited by 85
  19. 2024Improving 2D Feature Representations by 3D-Aware Fine-Tuningcited by 71
  20. 2022Interactive Segmentation of Radiance Fieldscited by 60
  21. 2020Semantic MapNet: Building Allocentric Semantic Maps and Representations from Egocentric Viewscited by 58
  22. 2018LSTA: Long Short-Term Attention for Egocentric Action Recognitioncited by 57
  23. 2021NeuralDiff: Segmenting 3D objects that move in egocentric videoscited by 57
  24. 2024Robot See Robot Do: Imitating Articulated Object Manipulation with Monocular 4D Reconstructioncited by 54
  25. 2023EPIC Fields: Marrying 3D Geometry and Video Understandingcited by 49
  26. 2016Going Deeper into First-Person Activity Recognitioncited by 48
  27. 2024EgoLifter: Open-world 3D Segmentation for Egocentric Perceptioncited by 47
  28. 2024Robo-GS: A Physics Consistent Spatial-Temporal Model for Robotic Arm with Hybrid Representationcited by 46
  29. 2024EgoVideo: Exploring Egocentric Foundation Model and Downstream Adaptationcited by 44
  30. 2016First-Person Activity Forecasting with Online Inverse Reinforcement Learningcited by 40
  31. 2024N2F2: Hierarchical Scene Understanding with Nested Neural Feature Fieldscited by 34
  32. 2024Spatial Cognition from Egocentric Video: Out of Sight, Not Out of Mindcited by 28
  33. 2022EgoLoc: Revisiting 3D Object Localization from Egocentric Videos with Visual Queriescited by 25
  34. 2024EgoGaussian: Dynamic Scene Understanding from Egocentric Video with 3D Gaussian Splattingcited by 23
  35. 2022EgoEnv: Human-centric environment representations from egocentric videocited by 22
  36. 2023Multi-label affordance mapping from egocentric visioncited by 18
  37. 2021UnweaveNet: Unweaving Activity Storiescited by 14
  38. 2024AFF-ttention! Affordances and Attention models for Short-Term Object Interaction Anticipationcited by 14

50 most cited of 53 All 53 (JSON)

Built on by

Not cited yet.