mlsanityData to 13 Oct 2026

SENSOR: Imitate Third-Person Expert's Behaviors via Active Sensoring

arXiv 2404.03386 ↗First posted 4 Apr 2024cs.RO cs.AI cs.LGcited by 1

On the map: Offline RL and imitation learning from data → Robot manipulation imitation learning

Abstract (arXiv)

In many real-world visual Imitation Learning (IL) scenarios, there is a misalignment between the agent's and the expert's perspectives, which might lead to the failure of imitation. Previous methods have generally solved this problem by domain alignment, which incurs extra computation and storage costs, and these methods fail to handle the textithard cases where the viewpoint gap is too large. To alleviate the above problems, we introduce active sensoring in the visual IL setting and propose a model-based SENSory imitatOR (SENSOR) to automatically change the agent's perspective to match the expert's. SENSOR jointly learns a world model to capture the dynamics of latent states, a sensor policy to control the camera, and a motor policy to control the agent. Experiments on visual locomotion tasks show that SENSOR can efficiently simulate the expert's perspective and strategy, and outperforms most baseline methods.

Key facts. Cited by 1 paper in the map (data through 13 Oct 2026). Placed in Robot manipulation imitation learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 32 of its 49 references that are in the map. First posted on arXiv 4 Apr 2024.

Builds on

32 of 49 references.

Closest ancestors · 2

Same direction, newest first

  1. 2021Self-Supervised Disentangled Representation Learning for Third-Person Imitation Learning2 Aug
  2. 2019Third-Person Visual Imitation Learning via Decoupled Hierarchical Controller21 Nov

Other references · 30

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,223
  2. 2014Unsupervised Domain Adaptation by Backpropagationcited by 1,909
  3. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,620
  4. 2016Generative Adversarial Imitation Learningcited by 1,577
  5. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,334
  6. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,241
  7. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,185
  8. 2018DeepMind Control Suitecited by 944
  9. 2018MINE: Mutual Information Neural Estimationcited by 682
  10. 2014Deep Domain Confusion: Maximizing for Domain Invariancecited by 597
  11. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 486
  12. 2018Human-level performance in first-person multiplayer games with population-based deep reinforcement learningcited by 260
18 more
  1. 2017One-Shot Visual Imitation Learning via Meta-Learningcited by 225
  2. 2017Deep Reinforcement Learning framework for Autonomous Drivingcited by 224
  3. 2019A Divergence Minimization Perspective on Imitation Learning Methodscited by 188
  4. 2016End-to-end Learning of Driving Models from Large-scale Video Datasetscited by 147
  5. 2018One-Shot Imitation from Observing Humans via Domain-Adaptive Meta-Learningcited by 146
  6. 2017Third-Person Imitation Learningcited by 103
  7. 2019Task-Relevant Adversarial Imitation Learningcited by 45
  8. 2021Cross-domain Imitation from Observationscited by 42
  9. 2021Visual Adversarial Imitation Learning using Variational Modelscited by 42
  10. 2016Model-based Adversarial Imitation Learningcited by 29
  11. 2018Reinforcement Learning of Active Vision for Manipulating Objects under Occlusionscited by 25
  12. 2021Domain-Robust Visual Imitation Learning with Mutual Information Constraintscited by 20
  13. 2019Adversarial Imitation Learning from Incomplete Demonstrationscited by 16
  14. 2022Learn what matters: cross-domain imitation learning with task-relevant embeddingscited by 14
  15. 2023SeMAIL: Eliminating Distractors in Visual Imitation via Separated Modelscited by 10
  16. 2020Distributed Reinforcement Learning of Targeted Grasping with Active Vision for Mobile Manipulatorscited by 7
  17. 2019Domain Adaptive Imitation Learningcited by 6
  18. 2023Active Third-Person Imitation Learningcited by 1

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025FOUNDER: Grounding Foundation Models in World Models for Open-Ended Embodied Decision Makingcited by 8

Similar, not cited

Close in wording, no citation either way

  1. 2026An Empirical Study on What Matters for Viewpoint-Generalizable Policies in Visual Imitation Learning
  2. 2025OSVI-WM: One-Shot Visual Imitation for Unseen Tasks using World-Model-Guided Trajectory Generation
  3. 2023Adversarial Imitation Learning from Visual Observations using Latent Information
  4. 2018Zero-Shot Visual Imitation
  5. 2026Mirror Learning
  6. 2021Manipulator-Independent Representations for Visual Imitation
  7. 2020Causal Transfer for Imitation Learning and Decision Making under Sensor-shift
  8. 2025Observer-Actor: Active Vision Imitation Learning with Sparse-View Gaussian Splatting