mlsanityData to 7 Oct 2026

Visionary: Vision architecture discovery for robot learning

arXiv 2103.14633 ↗First posted 26 Mar 2021cs.RO cs.CV cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

We propose a vision-based architecture search algorithm for robot manipulation learning, which discovers interactions between low dimension action inputs and high dimensional visual inputs. Our approach automatically designs architectures while training on the task - discovering novel ways of combining and attending image feature representations with actions as well as features from previous layers. The obtained new architectures demonstrate better task success rates, in some cases with a large margin, compared to a recent high performing baseline. Our real robot experiments also confirm that it improves grasping performance by 6%. This is the first approach to demonstrate a successful neural architecture search and attention connectivity search for a real-robot task.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 32 of its 40 references that are in the map. First posted on arXiv 26 Mar 2021.

Builds on

32 of 40 references.

References · 32

  1. 2017Squeeze-and-Excitation Networkscited by 4,763
  2. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  3. 2018DARTS: Differentiable Architecture Searchcited by 1,782
  4. 2016Neural Architecture Search with Reinforcement Learningcited by 1,489
  5. 2017Learning Transferable Architectures for Scalable Image Recognitioncited by 1,097
  6. 2018Regularized Evolution for Image Classifier Architecture Searchcited by 902
  7. 2015End-to-End Training of Deep Visuomotor Policiescited by 847
  8. 2017PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenescited by 768
  9. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 730
  10. 2017Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metricscited by 259
  11. 2015Supersizing Self-supervision: Learning to Grasp from 50K Tries and 700 Robot Hourscited by 208
  12. 2017Hierarchical Representations for Efficient Architecture Searchcited by 188
20 more
  1. 2018Making Sense of Vision and Touch: Self-Supervised Learning of Multimodal Representations for Contact-Rich Taskscited by 150
  2. 2017Robotic Pick-and-Place of Novel Objects in Clutter with Multi-Affordance Grasping and Cross-Domain Image Matchingcited by 124
  3. 2013Deep Learning for Detecting Robotic Graspscited by 114
  4. 2018Zero-Shot Visual Imitationcited by 114
  5. 2017Evolving Deep Neural Networkscited by 93
  6. 2019Learning to Manipulate Deformable Objects without Demonstrationscited by 76
  7. 2019AssembleNet: Searching for Multi-Stream Neural Connectivity in Video Architecturescited by 46
  8. 2019The Best of Both Modes: Separately Leveraging RGB and Depth for Unseen Object Instance Segmentationcited by 39
  9. 2016High precision grasp pose detection in dense cluttercited by 33
  10. 2018Deep Reinforcement Learning for Vision-Based Robotic Grasping: A Simulated Comparative Evaluation of Off-Policy Methodscited by 32
  11. 2019Mechanical Search: Multi-Step Retrieval of a Target Object Occluded by Cluttercited by 30
  12. 2017Learning a visuomotor controller for real world robotic grasping using simulated depth imagescited by 29
  13. 2020AssembleNet++: Assembling Modality Representations via Attention Connectionscited by 28
  14. 2019Tiny Video Networkscited by 27
  15. 2020Learning Precise 3D Manipulation from Multiple Uncalibrated Camerascited by 18
  16. 2018Evolving Space-Time Neural Architectures for Videoscited by 17
  17. 2019TextNAS: A Neural Architecture Search Space tailored for Text Representationcited by 16
  18. 2020An Introduction to Neural Architecture Search for Convolutional Networkscited by 10
  19. 2020X-Ray: Mechanical Search for an Occluded Object by Minimizing Support of Learned Occupancy Distributionscited by 7
  20. 2019Improving Data Efficiency of Self-supervised Learning for Robotic Graspingcited by 6

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 2; 2023: 1.

4 papers.

  1. 2022Automated Reinforcement Learning (AutoRL): A Survey and Open Problemscited by 69
  2. 2022Does Self-supervised Learning Really Improve Reinforcement Learning from Pixels?cited by 24
  3. 2021Differentiable Architecture Search for Reinforcement Learningcited by 4
  4. 2023HyperPPO: A scalable method for finding small policies for robotic controlcited by 4