mlsanityData to 14 Oct 2026

Q-attention: Enabling Efficient Learning for Vision-based Robotic Manipulation

arXiv 2105.14829 ↗First posted 31 May 2021cs.RO cs.AI cs.CVcited by 85

Not on the map. Why?

Cited by 21 in Offline RL and imitation learning from data, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Despite the success of reinforcement learning methods, they have yet to have their breakthrough moment when applied to a broad range of robotic manipulation tasks. This is partly due to the fact that reinforcement learning algorithms are notoriously difficult and time consuming to train, which is exacerbated when training from images rather than full-state inputs. As humans perform manipulation tasks, our eyes closely monitor every step of the process with our gaze focusing sequentially on the objects being manipulated. With this in mind, we present our Attention-driven Robotic Manipulation (ARM) algorithm, which is a general manipulation algorithm that can be applied to a range of sparse-rewarded tasks, given only a small number of demonstrations. ARM splits the complex task of manipulation into a 3 stage pipeline: (1) a Q-attention agent extracts relevant pixel locations from RGB and point cloud inputs, (2) a next-best pose agent that accepts crops from the Q-attention agent and outputs poses, and (3) a control agent that takes the goal pose and outputs joint actions. We show that current learning algorithms fail on a range of RLBench tasks, whilst ARM is successful.

Key facts. Cited by 85 papers in the map (data through 14 Oct 2026). Builds on 41 of its 54 references that are in the map. First posted on arXiv 31 May 2021.

Builds on

41 of 54 references.

References · 41

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  4. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,548
  5. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  6. 2016OpenAI Gymcited by 1,736
  7. 2016Generative Adversarial Imitation Learningcited by 1,597
  8. 2015Show, Attend and Tell: Neural Image Caption Generation with Visual Attentioncited by 1,423
  9. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,240
  10. 2018DeepMind Control Suitecited by 956
  11. 2015End-to-End Training of Deep Visuomotor Policiescited by 875
  12. 2018Self-Attention Generative Adversarial Networkscited by 809
29 more
  1. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 743
  2. 2017Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrationscited by 715
  3. 2019RLBench: The Robot Learning Benchmark & Learning Environmentcited by 689
  4. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 585
  5. 2019Learning agile and dynamic motor skills for legged robotscited by 504
  6. 2019DenseFusion: 6D Object Pose Estimation by Iterative Dense Fusioncited by 355
  7. 2020Transporter Networks: Rearranging the Visual World for Robotic Manipulationcited by 344
  8. 2017Overcoming Exploration in Reinforcement Learning with Demonstrationscited by 340
  9. 2017Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewardscited by 319
  10. 2016Deep Visual Foresight for Planning Robot Motioncited by 285
  11. 2019Improving Sample Efficiency in Model-Free Reinforcement Learning from Imagescited by 274
  12. 2017Asymmetric Actor Critic for Image-Based Robot Learningcited by 238
  13. 2016CAD2RL: Real Single-Image Flight without a Single Real Imagecited by 233
  14. 2018Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learningcited by 204
  15. 2018Sim-to-Real via Sim-to-Sim: Data-efficient Robotic Grasping via Randomized-to-Canonical Adaptation Networkscited by 174
  16. 2018Making Sense of Vision and Touch: Self-Supervised Learning of Multimodal Representations for Contact-Rich Taskscited by 154
  17. 2019TossingBot: Learning to Throw Arbitrary Objects with Residual Physicscited by 137
  18. 2018Sim-to-Real Reinforcement Learning for Deformable Object Manipulationcited by 131
  19. 2018Learning Synergies between Pushing and Grasping with Self-supervised Deep Reinforcement Learningcited by 128
  20. 2018Closing the Loop for Robotic Grasping: A Real-time, Generative Grasp Synthesis Approachcited by 120
  21. 2019Estimating Risk and Uncertainty in Deep Reinforcement Learningcited by 70
  22. 2017Transferring End-to-End Visuomotor Control from Simulation to Real World for a Multi-Stage Taskcited by 66
  23. 2020MoreFusion: Multi-object Reasoning for 6D Pose Estimation from Volumetric Fusioncited by 46
  24. 2019PyRep: Bringing V-REP to Deep Robot Learningcited by 44
  25. 2017Deep Predictive Policy Training using Reinforcement Learningcited by 32
  26. 2021Generalization Through Hand-Eye Coordination: An Action Space for Learning Spatially-Invariant Visuomotor Controlcited by 24
  27. 20163D Simulation for Robot Arm Control with Deep Q-Learningcited by 21
  28. 2020Tactical Decision-Making in Autonomous Driving by Reinforcement Learning with Uncertainty Estimationcited by 14
  29. 2019Perceptual Attention-based Predictive Controlcited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 1; 2022: 13; 2023: 19; 2024: 31; 2025: 13; 2026: 8.

85 papers.

Sort:

By area:

  1. 2022Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulationcited by 551
  2. 2022DayDreamer: World Models for Physical Robot Learningcited by 400
  3. 20243D Diffuser Actor: Policy Diffusion with 3D Scene Representationscited by 282
  4. 2022Real-World Robot Learning with Masked Visual Pre-trainingcited by 245
  5. 2022Masked World Models for Visual Controlcited by 170
  6. 2023Language-Driven Representation Learning for Roboticscited by 165
  7. 2024Reconciling Reality through Simulation: A Real-to-Sim-to-Real Approach for Robust Manipulationcited by 144
  8. 2023Act3D: 3D Feature Field Transformers for Multi-Task Robotic Manipulationcited by 138
  9. 2024THE COLOSSEUM: A Benchmark for Evaluating Generalization for Robotic Manipulationcited by 133
  10. 2021Coarse-to-Fine Q-attention: Efficient Learning for Visual Robotic Manipulation via Discretisationcited by 122
  11. 2022Instruction-driven history-aware policies for robotic manipulationscited by 94
  12. 2024Hierarchical Diffusion Policy for Kinematics-Aware Multi-Task Robotic Manipulationcited by 93
38 more
  1. 2024From Imitation to Refinement -- Residual RL for Precise Assemblycited by 90
  2. 2023Multi-View Masked World Models for Visual Robotic Manipulationcited by 80
  3. 2023Data Quality in Imitation Learningcited by 75
  4. 2022Auto-Lambda: Disentangling Dynamic Task Relationshipscited by 73
  5. 2023Waypoint-Based Imitation Learning for Robotic Manipulationcited by 61
  6. 2023Behavior Retrieval: Few-Shot Imitation Learning by Querying Unlabeled Datasetscited by 58
  7. 2023PolarNet: 3D Point Clouds for Language-Guided Robotic Manipulationcited by 56
  8. 2024Plan-Seq-Learn: Language Model Guided RL for Solving Long Horizon Robotics Taskscited by 52
  9. 2024RACER: Rich Language-Guided Failure Recovery Policies for Imitation Learningcited by 49
  10. 2025ReinboT: Amplifying Robot Visual-Language Manipulation with Reinforcement Learningcited by 47
  11. 2023KITE: Keypoint-Conditioned Policies for Semantic Manipulationcited by 39
  12. 2024Run-time Observation Interventions Make Vision-Language-Action Models More Visually Robustcited by 33
  13. 2025Exploring the Limits of Vision-Language-Action Manipulations in Cross-task Generalizationcited by 33
  14. 2022Frame Mining: a Free Lunch for Learning Robotic Manipulation from 3D Point Cloudscited by 32
  15. 2024BiGym: A Demo-Driven Mobile Bi-Manual Manipulation Benchmarkcited by 30
  16. 2023DexPBT: Scaling up Dexterous Manipulation for Hand-Arm Systems with Population Based Trainingcited by 29
  17. 2023Shelving, Stacking, Hanging: Relational Pose Diffusion for Multi-modal Rearrangementcited by 29
  18. 2024In-Context Learning Enables Robot Action Prediction in LLMscited by 27
  19. 2024Learning an Actionable Discrete Diffusion Policy via Large-Scale Actionless Video Pre-Trainingcited by 26
  20. 2023Universal Visual Decomposer: Long-Horizon Manipulation Made Easycited by 25
  21. 2022SEIL: Simulation-augmented Equivariant Imitation Learningcited by 23
  22. 2024Mitigating the Human-Robot Domain Discrepancy in Visual Pre-training for Robotic Manipulationcited by 23
  23. 2025UAD: Unsupervised Affordance Distillation for Generalization in Robotic Manipulationcited by 23
  24. 2023HarmonyDream: Task Harmonization Inside World Modelscited by 22
  25. 20253D FlowMatch Actor: Unified 3D Policy for Single- and Dual-Arm Manipulationcited by 20
  26. 2022On the Effectiveness of Fine-tuning Versus Meta-reinforcement Learningcited by 19
  27. 2024MALMM: Multi-Agent Large Language Models for Zero-Shot Robotics Manipulationcited by 19
  28. 2023Chain-of-Thought Predictive Controlcited by 18
  29. 2024One-Shot Imitation Learning with Invariance Matching for Robotic Manipulationcited by 18
  30. 2024SAM-E: Leveraging Visual Foundation Model with Sequence Imitation for Embodied Manipulationcited by 18
  31. 2024Imagination Policy: Using Generative Point Cloud Models for Learning Manipulation Policiescited by 18
  32. 2024Generative Image as Action Modelscited by 18
  33. 2024Continuous Control with Coarse-to-fine Reinforcement Learningcited by 17
  34. 2023A Universal Semantic-Geometric Representation for Robotic Manipulationcited by 16
  35. 2024Composing Pre-Trained Object-Centric Representations for Robotics From "What" and "Where" Foundation Modelscited by 16
  36. 2024Contrastive Imitation Learning for Language-guided Multi-Task Robotic Manipulationcited by 15
  37. 2024Scaling Robot Policy Learning via Zero-Shot Labeling with Foundation Modelscited by 14
  38. 2022Coarse-to-Fine Q-attention with Learned Path Rankingcited by 13

50 most cited of 85 All 85 (JSON)