mlsanityData to 7 Oct 2026

Privileged Sensing Scaffolds Reinforcement Learning

arXiv 2405.14853 ↗First posted 23 May 2024cs.LG cs.AI cs.ROcited by 18

Not on the map. Why?

Abstract (arXiv)

We need to look at our shoelaces as we first learn to tie them but having mastered this skill, can do it from touch alone. We call this phenomenon "sensory scaffolding": observation streams that are not needed by a master might yet aid a novice learner. We consider such sensory scaffolding setups for training artificial agents. For example, a robot arm may need to be deployed with just a low-cost, robust, general-purpose camera; yet its performance may improve by having privileged training-time-only access to informative albeit expensive and unwieldy motion capture rigs or fragile tactile sensors. For these settings, we propose "Scaffolder", a reinforcement learning approach which effectively exploits privileged sensing in critics, world models, reward estimators, and other such auxiliary components that are only used at training time, to improve the target policy. For evaluating sensory scaffolding agents, we design a new "S3" suite of ten diverse simulated robotic tasks that explore a wide range of practical sensor setups. Agents must use privileged camera sensing to train blind hurdlers, privileged active visual perception to help robot arms overcome visual occlusions, privileged touch sensors to train robot hands, and more. Scaffolder easily outperforms relevant prior baselines and frequently performs comparably even to policies that have test-time access to the privileged sensors. Website: https://penn-pal-lab.github.io/scaffolder/

Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). Builds on 33 of its 42 references that are in the map. First posted on arXiv 23 May 2024.

Builds on

33 of 42 references.

References · 33

  1. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
  2. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,401
  3. 2023Mastering Diverse Domains through World Modelscited by 1,193
  4. 2017Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrationscited by 685
  5. 2018Learning Dexterous In-Hand Manipulationcited by 643
  6. 2020Learning Quadrupedal Locomotion over Challenging Terraincited by 494
  7. 2017Time-Contrastive Networks: Self-Supervised Learning from Videocited by 453
  8. 2021RMA: Rapid Motor Adaptation for Legged Robotscited by 430
  9. 2019Learning by Cheatingcited by 361
  10. 2018Closing the Sim-to-Real Loop: Adapting Simulation Randomization with Real World Experiencecited by 203
  11. 2017Asymmetric Actor Critic for Image-Based Robot Learningcited by 202
  12. 2022In-Hand Object Rotation via Rapid Motor Adaptationcited by 102
21 more
  1. 2022Deep Hierarchical Planning from Pixelscited by 96
  2. 2023Multi-View Masked World Models for Visual Robotic Manipulationcited by 78
  3. 2018Variational Inverse Control with Events: A General Framework for Data-Driven Reward Definitioncited by 66
  4. 2022Watch and Match: Supercharging Imitation with Regularized Optimal Transportcited by 64
  5. 2023Sequential Dexterity: Chaining Dexterous Policies for Long-Horizon Manipulationcited by 43
  6. 2020Bridging the Imitation Gap by Adaptive Insubordinationcited by 38
  7. 2022Sequence Model Imitation Learning with Unobserved Contextscited by 34
  8. 2023RoboPianist: Dexterous Piano Playing with Deep Reinforcement Learningcited by 29
  9. 2016Visual Closed-Loop Control for Pouring Liquidscited by 26
  10. 2021Unbiased Asymmetric Reinforcement Learning under Partial Observabilitycited by 24
  11. 2022Transfer RL across Observation Feature Spaces via Model-Based Regularizationcited by 18
  12. 2022Leveraging Fully Observable Policies for Learning under Partial Observabilitycited by 17
  13. 2023TGRL: An Algorithm for Teacher Guided Reinforcement Learningcited by 16
  14. 2021Policy Transfer across Visual and Dynamics Domain Gaps via Iterative Groundingcited by 12
  15. 2023Informed POMDP: Leveraging Additional Information in Model-Based RLcited by 11
  16. 2020Privileged Information Dropout in Reinforcement Learningcited by 8
  17. 2022Learning to Brachiate via Simplified Model Imitationcited by 7
  18. 2022Transferring Implicit Knowledge of Non-Visual Object Properties Across Heterogeneous Robot Morphologiescited by 7
  19. 2019Simultaneously Learning Vision and Feature-based Control Policies for Real-world Ball-in-a-Cupcited by 6
  20. 2021Reinforcement Learning using Guided Observabilitycited by 3
  21. 2022Training Robots to Evaluate Robots: Example-Based Interactive Reward Functions for Policy Learningcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 10; 2026: 6.

18 papers.

Sort:

By area:

  1. 2024Better than Your Teacher: LLM Agents that learn from Privileged AI Feedbackcited by 21
  2. 2024Student-Informed Teacher Trainingcited by 6
  3. 2025SkyDreamer: Interpretable End-to-End Vision-Based Drone Racing with Model-Based Reinforcement Learningcited by 5
  4. 2025Multi-Objective Reinforcement Learning for Adaptable Personalized Autonomous Drivingcited by 3
  5. 2025Privileged observations enable rapid and reliable policy discovery directly in the physical worldcited by 3
  6. 2025Merging and Disentangling Views in Visual Reinforcement Learning for Robotic Manipulationcited by 2
  7. 2025Distilling Realizable Students from Unrealizable Teacherscited by 2
  8. 2025Informed Asymmetric Actor-Critic: Leveraging Privileged Signals Beyond Full-State Accesscited by 2
  9. 2026Squint: Fast Visual Reinforcement Learning for Sim-to-Real Roboticscited by 2
  10. 2025No More Blind Spots: Learning Vision-Based Omnidirectional Bipedal Locomotion for Challenging Terraincited by 1
  11. 2025Unifying Agent Interaction and World Information for Multi-agent Coordinationcited by 1
  12. 2025Real-World Reinforcement Learning of Active Perception Behaviorscited by 1
6 more
  1. 2026JEPA-x: Cross-Predictive Physics Grounding for Forecastable Latent Dynamicscited by 1
  2. 2025Learn A Flexible Exploration Model for Parameterized Action Markov Decision Processescited by 0
  3. 2026Reinformed Dreamer: An Asymmetric World Model Efficiently Trained through Latent Guidancecited by 0
  4. 2026Shared Actors Need Not Share Critics: Effects of Value Mismatch in Parallel Reinforcement Learningcited by 0
  5. 2026Toward Interactive Understanding of Code APIscited by 0
  6. 2026Action Shaping: Policies Absorb What They Can Expresscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026LEAP: Making Privileged Geometry Supervision Effective for Visuomotor Learning ↗
  2. 2026Artificial Foveated Perception for Mitigating Shortcut Learning in Robotic Foundation Models
  3. 2022Learning Sensorimotor Primitives of Sequential Manipulation Tasks from Visual Demonstrations
  4. 2025Scaffolding Dexterous Manipulation with Vision-Language Models
  5. 2023Privileged to Predicted: Towards Sensorimotor Reinforcement Learning for Urban Driving
  6. 2024Low Fidelity Visuo-Tactile Pretraining Improves Vision-Only Manipulation Performance
  7. 2024Pushing the Limits of Cross-Embodiment Learning for Manipulation and Navigation
  8. 2026Improving Vision-Language-Action Model Fine-Tuning with Structured Stage and Keyframe Supervision