mlsanityData to 14 Oct 2026

Multi-Instance Aware Localization for End-to-End Imitation Learning

arXiv 2101.01053 ↗First posted 26 Dec 2020cs.RO cs.LGcited by 0

On the map: Offline RL and imitation learning from data → Robot manipulation imitation learning

Abstract (arXiv)

Existing architectures for imitation learning using image-to-action policy networks perform poorly when presented with an input image containing multiple instances of the object of interest, especially when the number of expert demonstrations available for training are limited. We show that end-to-end policy networks can be trained in a sample efficient manner by (a) appending the feature map output of the vision layers with an embedding that can indicate instance preference or take advantage of an implicit preference present in the expert demonstrations, and (b) employing an autoregressive action generator network for the control layers. The proposed architecture for localization has improved accuracy and sample efficiency and can generalize to the presence of more instances of objects than seen during training. When used for end-to-end imitation learning to perform reach, push, and pick-and-place tasks on a real robot, training is achieved with as few as 15 expert demonstrations.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Robot manipulation imitation learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 16 of its 22 references that are in the map. First posted on arXiv 26 Dec 2020.

Builds on

16 of 22 references.

Closest ancestors · 1

Same direction, newest first

  1. 2018One-Shot Hierarchical Imitation Learning of Compound Visuomotor Tasks25 Oct

Other references · 15

  1. 2017Attention Is All You Needcited by 60,361
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  4. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  5. 2015End-to-End Training of Deep Visuomotor Policiescited by 875
  6. 2018An Intriguing Failing of Convolutional Neural Networks and the CoordConv Solutioncited by 307
  7. 2013DeepPose: Human Pose Estimation via Deep Neural Networkscited by 305
  8. 2017Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperationcited by 289
  9. 2019Attentive Neural Processescited by 283
  10. 2017One-Shot Visual Imitation Learning via Meta-Learningcited by 228
  11. 2013OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networkscited by 224
  12. 2018One-Shot Imitation from Observing Humans via Domain-Adaptive Meta-Learningcited by 148
3 more
  1. 2017Robotic Pick-and-Place of Novel Objects in Clutter with Multi-Affordance Grasping and Cross-Domain Image Matchingcited by 125
  2. 2017Deep Reinforcement Learning for High Precision Assembly Taskscited by 39
  3. 2017Attention-based Extraction of Structured Information from Street View Imagerycited by 14

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023Memory-Consistent Neural Networks for Imitation Learning
  2. 2026Difference-Aware Retrieval Policies for Imitation Learning
  3. 2021Towards More Generalizable One-shot Visual Imitation Learning
  4. 2021Coarse-to-Fine Imitation Learning: Robot Manipulation from a Single Demonstration
  5. 2026Multi-Camera View Scaling for Data-Efficient Robot Imitation Learning
  6. 2022Practical Imitation Learning in the Real World via Task Consistency Loss
  7. 2023One-Shot Imitation Learning: A Pose Estimation Perspective
  8. 2021The Surprising Effectiveness of Representation Learning for Visual Imitation