LIME: Learning Intent-aware Camera Motion from Egocentric Video
Not on the map. Why?
Abstract (arXiv)
Autonomous robots often need to move their camera before they can act: to inspect an object, reveal an occluded region, or obtain a view that responds to a user's intent. While vision-language navigation translates instructions to base motion and vision-language-action policies map instructions to manipulation actions, language-conditioned camera motion remains comparatively underexplored as a first-class action. We formulate language-conditioned camera motion generation: given a current RGB observation and a free-form natural-language intent, predict a relative target camera pose for the next observation. This task is inherently non-trivial: viewpoint changes are driven by latent perceptual intentions, and a valid motion may operate at different semantic granularity, from entering a room to looking around a corner, inspecting a visible object, or revealing an occluded detail. To model this structure, we mine multi-intention camera-motion supervision from egocentric video, pairing plausible intents and observation-gain descriptions with relative SE(3) target poses. We propose LIME, a vision-language camera-motion generator that combines an auto-regressive observation-gain output with a continuous flow-matching pose head. This design lets the model jointly predict what the next view should reveal while representing multi-hypothesis target views. Across experiments and downstream robotic tasks, we show that LIME can learn to actively choose camera poses from passive human video, turning ordinary egocentric recordings into supervision for intent-aware active perception.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 58 of its 69 references that are in the map. First posted on arXiv 2 Jul 2026.
Builds on
58 of 69 references.
References · 58
- 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
- 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
- 2025Qwen3-VL Technical Reportcited by 2,534
- 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,297
- 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,653
- 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,338
- 2018On the Continuity of Rotation Representations in Neural Networkscited by 1,186
- 2023ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenescited by 747
- 2025Depth Anything 3: Recovering the Visual Space from Any Viewscited by 626
- 2023Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectivescited by 477
- 2020Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environmentscited by 429
- 2017Embodied Question Answeringcited by 335
38 more
- 2020A Recurrent Vision-and-Language BERT for Navigationcited by 300
- 2025MoGe-2: Accurate Monocular Geometry with Metric Scale and Sharp Detailscited by 250
- 2020ObjectNav Revisited: On Evaluation of Embodied Agents Navigating to Objectscited by 240
- 2020The EPIC-KITCHENS Dataset: Collection, Challenges and Baselinescited by 221
- 2024NaVILA: Legged Robot Vision-Language-Action Model for Navigationcited by 201
- 2024EgoMimic: Scaling Imitation Learning via Egocentric Videocited by 160
- 2024InstructNav: Zero-shot System for Generic Instruction Navigation in Unexplored Environmentcited by 150
- 2022Vision-and-Language Navigation: A Survey of Tasks, Methods, and Future Directionscited by 116
- 2024Nymeria: A Massive Collection of Multimodal Egocentric Daily Motion in the Wildcited by 109
- 2025StreamVLN: Streaming Vision-and-Language Navigation via SlowFast Context Modelingcited by 107
- 2024GOAT-Bench: A Benchmark for Multi-Modal Lifelong Navigationcited by 93
- 2016Revisiting Active Perceptioncited by 90
- 2023GOAT: GO to Any Thingcited by 89
- 2025Vision-Language-Action Models for Robotics: A Review Towards Real-World Applicationscited by 82
- 2025JanusVLN: Decoupling Semantics and Spatiality with Dual Implicit Memory for Vision-Language Navigationcited by 80
- 2024Vision-and-Language Navigation Today and Tomorrow: A Survey in the Era of Foundation Modelscited by 79
- 2026EgoScale: Scaling Dexterous Manipulation with Diverse Egocentric Human Datacited by 77
- 2024Explore until Confident: Efficient Exploration for Embodied Question Answeringcited by 74
- 2024HM3D-OVON: A Dataset and Benchmark for Open-Vocabulary Object Goal Navigationcited by 71
- 2022A Survey on Active Simultaneous Localization and Mapping: State of the Art and New Frontierscited by 62
- 2025UniGoal: Towards Universal Zero-shot Goal-oriented Navigationcited by 56
- 2025Ground Slow, Move Fast: A Dual-System Foundation Model for Generalizable Vision-and-Language Navigationcited by 56
- 2021SEAL: Self-supervised Embodied Active Learning using Exploration and 3D Consistencycited by 47
- 20223D-Aware Object Goal Navigation via Simultaneous Exploration and Identificationcited by 46
- 2025VidBot: Learning Generalizable 3D Actions from In-the-Wild 2D Human Videos for Zero-Shot Robotic Manipulationcited by 46
- 2019An Efficient Sampling-based Method for Online Informative Path Planning in Unknown Environmentscited by 38
- 2024End-to-End Navigation with Vision Language Models: Transforming Spatial Reasoning into Question-Answeringcited by 37
- 2025Vision in Action: Learning Active Perception from Human Demonstrationscited by 33
- 2025Beyond the Destination: A Novel Benchmark for Exploration-Aware Embodied Question Answeringcited by 29
- 2024RoomTour3D: Geometry-Aware Video-Instruction Tuning for Embodied Navigationcited by 23
- 2025Eye, Robot: Learning to Look to Act with a BC-RL Perception-Action Loopcited by 23
- 2026ABot-N0: Technical Report on the VLA Foundation Model for Versatile Embodied Navigationcited by 23
- 2026ActiveVLA: Injecting Active Perception into Vision-Language-Action Models for Precise 3D Robotic Manipulationcited by 14
- 2025FrontierNet: Learning Visual Cues to Explorecited by 11
- 2023Frontier Semantic Exploration for Visual Target Navigationcited by 9
- 2025Toward Ambulatory Vision: Learning Visually-Grounded Active View Selectioncited by 5
- 2026ActiveGlasses: Learning Manipulation with Active Vision from Ego-centric Human Demonstrationcited by 5
- 2022Active SLAM: A Review On Last Decadecited by 4
50 most cited of 58 All 58 (JSON)
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2026World-Calibrated Proposal-to-Action Flow for Vision-Language-Action Models
- 2026Being-H0.7: A Latent World-Action Model from Egocentric Videos
- 2026EgoActor: Grounding Task Planning into Spatial-aware Egocentric Actions for Humanoid Robots via Visual-Language Models
- 2025Look, Zoom, Understand: The Robotic Eyeball for Embodied Perception
- 2026UniviewVLA: A Unified Multiview Vision-Language-Action Model with World Modeling
- 2026Ego-Human Motion Prediction with 3D-Aware LLM
- 2024Simultaneous Localization and Affordance Prediction of Tasks from Egocentric Video
- 2026SaPaVe: Towards Active Perception and Manipulation in Vision-Language-Action Models for Robotics