mlsanityData to 14 Oct 2026

INSPECT: Learning Robot View Selection from Assistant Use

arXiv 2609.20615 ↗First posted 17 Sep 2026cs.RO cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Robots inspecting an assembly must determine which parts are present and whether they are correctly installed. During egocentric assembly assistance, head motion and workpiece handling reveal evidence for these checks, while spoken state confirmations link observations to procedural outcomes. We introduce INSPECT, which learns robot view preferences from records of a smart-glasses assistant that answers part queries and provides next-step guidance. Presence-Invariant TwinSwap (PI-TwinSwap) calibrates object evidence through paired identity interventions. Claim-indexed supervision separates evidence requirements from camera-reproducible observation changes. Object-centered calibration adapts relative view preferences to robot poses, while clause-level screening checks predicted evidence. The robot selects views using only its current observation and known poses, without candidate images. Evaluation uses annotated assistant-video replay to simulate state feedback, without target-domain view labels for policy training. On images of physical gearbox assemblies, INSPECT achieves the highest view utility among the compared non-oracle policies and raises human-rated full verifiability from 34.8% to 41.7% compared with keeping the current view. On commercial angle-grinder recordings in IMPACT, the transferred relative-view selector increases the correct decision rate from 50.6% to 54.3% with a frozen perception head. The source code is available at https://github.com/Kratos-Wen/INSPECT.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 43 of its 49 references that are in the map. First posted on arXiv 17 Sep 2026.

Builds on

43 of 49 references.

References · 43

  1. 2025Qwen3-VL Technical Reportcited by 2,793
  2. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,556
  3. 2020Rescaling Egocentric Visioncited by 540
  4. 2023VideoMAE V2: Scaling Video Masked Autoencoders with Dual Maskingcited by 528
  5. 2023Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectivescited by 490
  6. 2019SelectiveNet: A Deep Neural Network with an Integrated Reject Optioncited by 350
  7. 2017Embodied Question Answeringcited by 336
  8. 2022BoT-SORT: Robust Associations Multi-Pedestrian Trackingcited by 320
  9. 2023Robots That Ask For Help: Uncertainty Alignment for Large Language Model Plannerscited by 308
  10. 2022Assembly101: A Large-Scale Multi-View Video Dataset for Understanding Procedural Activitiescited by 287
  11. 2025MoGe-2: Accurate Monocular Geometry with Metric Scale and Sharp Detailscited by 261
  12. 2023MimicPlay: Long-Horizon Imitation Learning by Watching Human Playcited by 223
31 more
  1. 2024EgoMimic: Scaling Imitation Learning via Egocentric Videocited by 175
  2. 2023HoloAssist: an Egocentric Human Interaction Dataset for Interactive AI Assistants in the Real Worldcited by 166
  3. 2026Ultralytics YOLO26: Unified Real-Time End-to-End Vision Modelscited by 57
  4. 2024Vid2Robot: End-to-end Video-conditioned Policy Learning with Cross-Attention Transformerscited by 52
  5. 2025Can We Detect Failures Without Failure Data? Uncertainty-Aware Runtime Failure Detection for Imitation Learning Policiescited by 52
  6. 2023IndustReal: A Dataset for Procedure Step Recognition Handling Execution Errors in Egocentric Videos in an Industrial-Like Settingcited by 48
  7. 2024PREGO: online mistake detection in PRocedural EGOcentric videoscited by 39
  8. 2025Vision in Action: Learning Active Perception from Human Demonstrationscited by 37
  9. 2025Active Video Perception: Iterative Evidence Seeking for Agentic Long Video Understandingcited by 29
  10. 2022Closed-Loop Next-Best-View Planning for Target-Driven Graspingcited by 19
  11. 2025ActiveUMI: Robotic Manipulation with Active Perception from Robot-Free Human Demonstrationscited by 19
  12. 2026ActiveVLA: Injecting Active Perception into Vision-Language-Action Models for Precise 3D Robotic Manipulationcited by 18
  13. 2025VISO-Grasp: Vision-Language Informed Spatial Object-centric 6-DoF Active View Planning and Grasping in Clutter and Invisibilitycited by 11
  14. 2023Active-Perceptive Motion Generation for Mobile Manipulationcited by 10
  15. 2025MICA: Multi-Agent Industrial Coordination Assistantcited by 8
  16. 2026SaPaVe: Towards Active Perception and Manipulation in Vision-Language-Action Models for Roboticscited by 8
  17. 2026IMPACT: A Dataset for Multi-Granularity Human Procedural Action Understanding in Industrial Assemblycited by 8
  18. 2026Plan, Watch, Recover: A Benchmark and Architectures for Proactive Procedural Assistancecited by 7
  19. 2024Scene Exploration by Vision-Language Modelscited by 6
  20. 2025Snap, Segment, Deploy: A Visual Data and Detection Pipeline for Wearable Industrial Assistantscited by 6
  21. 2025Toward Ambulatory Vision: Learning Visually-Grounded Active View Selectioncited by 6
  22. 2026ActiveMimic: Egocentric Video Pretraining with Active Perceptioncited by 6
  23. 2025Building Egocentric Procedural AI Assistant: Methods, Benchmarks, and Challengescited by 5
  24. 2025Multi-Keypoint Affordance Representation for Functional Dexterous Graspingcited by 4
  25. 2026Pro2Assist: Continuous Step-aware Proactive Assistance with Multi-modal Egocentric Perception for Long-horizon Procedural Taskscited by 3
  26. 2026OA-NBV: Occlusion-Aware Next-Best-View Planning for Human-Centered Active Perception on Mobile Robotscited by 2
  27. 2026Robot Planning and Situation Handling with Active Perceptioncited by 2
  28. 2026LIME: Learning Intent-aware Camera Motion from Egocentric Videocited by 2
  29. 2026EgoProceVQA: A Novel Egocentric Procedural Understanding Task with Self-Skill-Exploration Agentcited by 2
  30. 2021Introspective Visuomotor Control: Exploiting Uncertainty in Deep Visuomotor Control for Failure Recoverycited by 1
  31. 2026Hierarchical Policies from Verbal and Egocentric Human Signals for Natural Human-Robot Interactioncited by 1

Built on by

Not cited yet.