mlsanityData to 14 Oct 2026

Coarse-to-Fine Q-attention: Efficient Learning for Visual Robotic Manipulation via Discretisation

arXiv 2106.12534 ↗First posted 23 Jun 2021cs.RO cs.AI cs.CVcited by 122

Not on the map. Why?

Abstract (arXiv)

We present a coarse-to-fine discretisation method that enables the use of discrete reinforcement learning approaches in place of unstable and data-inefficient actor-critic methods in continuous robotics domains. This approach builds on the recently released ARM algorithm, which replaces the continuous next-best pose agent with a discrete one, with coarse-to-fine Q-attention. Given a voxelised scene, coarse-to-fine Q-attention learns what part of the scene to 'zoom' into. When this 'zooming' behaviour is applied iteratively, it results in a near-lossless discretisation of the translation space, and allows the use of a discrete action, deep Q-learning method. We show that our new coarse-to-fine algorithm achieves state-of-the-art performance on several difficult sparsely rewarded RLBench vision-based robotics tasks, and can train real-world policies, tabula rasa, in a matter of minutes, with as little as 3 demonstrations.

Key facts. Cited by 122 papers in the map (data through 14 Oct 2026). Builds on 33 of its 46 references that are in the map. First posted on arXiv 23 Jun 2021.

Builds on

33 of 46 references.

References · 33

  1. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2014Going Deeper with Convolutionscited by 4,241
  4. 2015Continuous control with deep reinforcement learningcited by 3,407
  5. 2015Trust Region Policy Optimizationcited by 2,580
  6. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  7. 2016Generative Adversarial Imitation Learningcited by 1,597
  8. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,240
  9. 2017Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrationscited by 715
  10. 2019RLBench: The Robot Learning Benchmark & Learning Environmentcited by 689
  11. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 585
  12. 2020Transporter Networks: Rearranging the Visual World for Robotic Manipulationcited by 344
21 more
  1. 2019Improving Sample Efficiency in Model-Free Reinforcement Learning from Imagescited by 274
  2. 2017Asymmetric Actor Critic for Image-Based Robot Learningcited by 238
  3. 2021MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scalecited by 229
  4. 2018Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learningcited by 204
  5. 2018Sim-to-Real Reinforcement Learning for Deformable Object Manipulationcited by 131
  6. 2018Learning Synergies between Pushing and Grasping with Self-supervised Deep Reinforcement Learningcited by 128
  7. 2018Closing the Loop for Robotic Grasping: A Real-time, Generative Grasp Synthesis Approachcited by 120
  8. 2019Grasping in the Wild:Learning 6DoF Closed-Loop Grasping from Low-Cost Demonstrationscited by 107
  9. 2021Volumetric Grasping Network: Real-time 6 DOF Grasp Detection in Cluttercited by 95
  10. 2021Q-attention: Enabling Efficient Learning for Vision-based Robotic Manipulationcited by 85
  11. 2018Composable Deep Reinforcement Learning for Robotic Manipulationcited by 72
  12. 2018Task-Embedded Control Networks for Few-Shot Imitation Learningcited by 66
  13. 2020MoreFusion: Multi-object Reasoning for 6D Pose Estimation from Volumetric Fusioncited by 46
  14. 2019Learning One-Shot Imitation from Humans without Humanscited by 33
  15. 20163D Simulation for Robot Arm Control with Deep Q-Learningcited by 21
  16. 2019Object Finding in Cluttered Scenes Using Interactive Perceptioncited by 18
  17. 2018Learning 6-DoF Grasping and Pick-Place Using Attention Focuscited by 7
  18. 2021Ivy: Templated Deep Learning for Inter-Framework Portabilitycited by 6
  19. 2019Pixel-Attentive Policy Gradient for Multi-Fingered Grasping in Cluttered Scenescited by 3
  20. 2019Learning Manipulation Skills Via Hierarchical Spatial Attentioncited by 3
  21. 2020Probabilistic 3D Multilabel Real-time Mapping for Multi-object Manipulationcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 1; 2022: 14; 2023: 18; 2024: 42; 2025: 34; 2026: 13.

122 papers.

Sort:

By area:

  1. 2022Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulationcited by 551
  2. 2022DayDreamer: World Models for Physical Robot Learningcited by 400
  3. 20243D Diffuser Actor: Policy Diffusion with 3D Scene Representationscited by 282
  4. 2022Real-World Robot Learning with Masked Visual Pre-trainingcited by 245
  5. 2023RVT: Robotic View Transformer for 3D Object Manipulationcited by 225
  6. 2022Masked World Models for Visual Controlcited by 170
  7. 2023Language-Driven Representation Learning for Roboticscited by 165
  8. 2023Structured World Models from Human Videoscited by 147
  9. 2024Reconciling Reality through Simulation: A Real-to-Sim-to-Real Approach for Robust Manipulationcited by 144
  10. 2023Act3D: 3D Feature Field Transformers for Multi-Task Robotic Manipulationcited by 138
  11. 2024RVT-2: Learning Precise Manipulation from Few Demonstrationscited by 137
  12. 2024THE COLOSSEUM: A Benchmark for Evaluating Generalization for Robotic Manipulationcited by 133
38 more
  1. 2024ManiGaussian: Dynamic Gaussian Splatting for Multi-task Robotic Manipulationcited by 108
  2. 2023GNFactor: Multi-Task Real Robot Learning with Generalizable Neural Feature Fieldscited by 99
  3. 2022Instruction-driven history-aware policies for robotic manipulationscited by 94
  4. 2024Hierarchical Diffusion Policy for Kinematics-Aware Multi-Task Robotic Manipulationcited by 93
  5. 2024LLARVA: Vision-Action Instruction Tuning Enhances Robot Learningcited by 87
  6. 2023Multi-View Masked World Models for Visual Robotic Manipulationcited by 80
  7. 2025BridgeVLA: Input-Output Alignment for Efficient 3D Manipulation Learning with Vision-Language Modelscited by 80
  8. 2024ManiCM: Real-time 3D Diffusion Policy via Consistency Model for Robotic Manipulationcited by 57
  9. 2023PolarNet: 3D Point Clouds for Language-Guided Robotic Manipulationcited by 56
  10. 20254D-VLA: Spatiotemporal Vision-Language-Action Pretraining with Cross-Scene Calibrationcited by 56
  11. 2024Plan-Seq-Learn: Language Model Guided RL for Solving Long Horizon Robotics Taskscited by 52
  12. 2022Look Closer: Bridging Egocentric and Third-Person Views with Transformers for Robotic Manipulationcited by 49
  13. 2024RACER: Rich Language-Guided Failure Recovery Policies for Imitation Learningcited by 49
  14. 2024Dream to Manipulate: Compositional World Models Empowering Robot Imitation Learning with Imaginationcited by 45
  15. 2023ProxyFormer: Proxy Alignment Assisted Point Cloud Completion with Missing Part Sensitive Transformercited by 43
  16. 2024Towards Generalizable Vision-Language Robotic Manipulation: A Benchmark and LLM-guided 3D Policycited by 38
  17. 2024Deep Generative Models in Robotics: A Survey on Learning from Multimodal Demonstrationscited by 35
  18. 2024Fourier Transporter: Bi-Equivariant Robotic Manipulation in 3Dcited by 33
  19. 2025FP3: A 3D Foundation Policy for Robotic Manipulationcited by 32
  20. 2024BiGym: A Demo-Driven Mobile Bi-Manual Manipulation Benchmarkcited by 30
  21. 2025LLaDA-VLA: Vision Language Diffusion Action Modelscited by 30
  22. 2023Shelving, Stacking, Hanging: Relational Pose Diffusion for Multi-modal Rearrangementcited by 29
  23. 2023AR2-D2:Training a Robot Without a Robotcited by 27
  24. 2024Autoregressive Action Sequence Learning for Robotic Manipulationcited by 27
  25. 2025Pre-training Auto-regressive Robotic Models with 4D Representationscited by 27
  26. 2024Learning an Actionable Discrete Diffusion Policy via Large-Scale Actionless Video Pre-Trainingcited by 26
  27. 2024SUGAR: Pre-training 3D Visual Representations for Roboticscited by 26
  28. 2024ActionFlow: Equivariant, Accurate, and Efficient Policies with Spatially Symmetric Flow Matchingcited by 26
  29. 2024GenDP: 3D Semantic Fields for Category-Level Generalizable Diffusion Policycited by 25
  30. 2022SEIL: Simulation-augmented Equivariant Imitation Learningcited by 23
  31. 2024DNAct: Diffusion Guided Multi-Task 3D Policy Learningcited by 23
  32. 2025UAD: Unsupervised Affordance Distillation for Generalization in Robotic Manipulationcited by 23
  33. 2023HarmonyDream: Task Harmonization Inside World Modelscited by 22
  34. 2024VoxAct-B: Voxel-Based Acting and Stabilizing Policy for Bimanual Manipulationcited by 21
  35. 2024Render and Diffuse: Aligning Image and Action Spaces for Diffusion-based Behaviour Cloningcited by 20
  36. 2022On the Effectiveness of Fine-tuning Versus Meta-reinforcement Learningcited by 19
  37. 2024MALMM: Multi-Agent Large Language Models for Zero-Shot Robotics Manipulationcited by 19
  38. 2024One-Shot Imitation Learning with Invariance Matching for Robotic Manipulationcited by 18

50 most cited of 122 All 122 (JSON)