mlsanityData to 14 Oct 2026

Learning Object Manipulation from Scratch via Contrastive Interaction

arXiv 2606.11525 ↗First posted 10 Jun 2026cs.RO cs.LGcited by 0

On the map: Deep reinforcement learning methods → RL for robot locomotion and manipulation

Abstract (arXiv)

Contrastive Reinforcement Learning (CRL) has seen recent success in a wide variety of goal-conditioned robotics tasks by learning structured representations of the dynamics. However, despite its success in locomotion and simpler control domains, CRL often struggles in interaction-rich manipulation. We argue that a key source of this difficulty is object-centric interaction, such as contact or grasping, that induces distinct changes in the underlying dynamic modes. In this work, we formulate manipulation dynamics as a piecewise-smooth Markov process and show that interaction-induced mode changes create piecewise nonlinear reachability structures that are difficult for standard CRL energy functions to represent and plan over. Based on this analysis, we introduce Interaction-weighted Resampling (IWR). IWR performs interaction-aware resampling around phases before, during, and after interactions, encouraging the learned representation to preserve the mode boundaries that determine future reachability to capture multi-modal and piecewise nonlinear reachability. Across interaction-centric environments, including 2D dynamic control, robotic manipulation, and robot air hockey, IWR improves both sample efficiency and overall performance over prior CRL methods, with 19.8% average improvement in simulation. Finally, using a sim-to-real pipeline with policies trained by IWR, we demonstrate the first real-world goal-conditioned robot air hockey agent capable of hitting goals, improving success from 25% to 60%. Project Page: IWR-arxiv.github.io.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods. Builds on 32 of its 93 references that are in the map. First posted on arXiv 10 Jun 2026.

Builds on

32 of 93 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Robot Air Hockey: A Manipulation Testbed for Robot Learning with Reinforcement Learning6 May

Other references · 31

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  3. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
  4. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,498
  5. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 752
  6. 2017Sim-to-Real Transfer of Robotic Control with Dynamics Randomizationcited by 672
  7. 2021RMA: Rapid Motor Adaptation for Legged Robotscited by 508
  8. 2020Learning Invariant Representations for Reinforcement Learning without Reconstructioncited by 436
  9. 2017Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewardscited by 319
  10. 2022Goal-Conditioned Reinforcement Learning: Problems and Solutionscited by 158
  11. 2024AdaptiGraph: Material-Adaptive Graph-Based Neural Dynamics for Robotic Manipulationcited by 54
  12. 2026Temporal Straightening for Latent Planningcited by 38
19 more
  1. 2025Particle-Grid Neural Dynamics for Learning Deformable Object Models from RGB-D Videoscited by 25
  2. 2024Proto Successor Measure: Representing the Behavior Space of an RL Agentcited by 20
  3. 2026InfoNCE Induces Gaussian Distributioncited by 16
  4. 2026Latent Particle World Models: Self-supervised Object-centric Stochastic Dynamics Modelingcited by 15
  5. 2024Fine-Grained Causal Dynamics Learning with Quantization for Improving Robustness in Reinforcement Learningcited by 11
  6. 2025Multistep Quasimetric Learning for Scalable Goal-conditioned Reinforcement Learningcited by 8
  7. 2024Learning Action-based Representations Using Invariancecited by 7
  8. 2026Hierarchical Entity-centric Reinforcement Learning with Factored Subgoal Diffusioncited by 7
  9. 2026Factored Latent Action World Modelscited by 7
  10. 2026Compositional Planning with Jumpy World Modelscited by 7
  11. 2023Granger Causal Interaction Skill Chainscited by 6
  12. 2025Null Counterfactual Factor Interactions for Goal-Conditioned Reinforcement Learningcited by 6
  13. 2025Demystifying the Mechanisms Behind Emergent Exploration in Goal-conditioned RLcited by 5
  14. 2024Automated Discovery of Functional Actual Causes in Complex Environmentscited by 4
  15. 2025From Pixels to Factors: Learning Independently Controllable State Variables for Reinforcement Learningcited by 3
  16. 2026Can We Really Learn One Representation to Optimize All Rewards?cited by 3
  17. 2026Unifying Goal-Conditioned RL and Unsupervised Skill Learning via Control-Maximizationcited by 2
  18. 2024Enhancing Contrastive Learning with Efficient Combinatorial Positive Pairingcited by 1
  19. 2026SUSD: Structured Unsupervised Skill Discovery through State Factorizationcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025FLEX: A Framework for Learning Robot-Agnostic Force-based Skills Involving Sustained Contact Object Manipulation
  2. 2025Prior Reinforce: Goal-Conditioned Dynamic Manipulation with Limited Trials ↗
  3. 2026Combined Constrained Sampling and Reinforcement Learning for Robotic Manipulation
  4. 2021Correct Me if I am Wrong: Interactive Learning for Robotic Manipulation
  5. 2026Physics-informed Goal-Conditioned Reinforcement Learning under Hybrid Contact Dynamics
  6. 2026Tune to Learn: How Controller Gains Shape Robot Policy Learning
  7. 2026One-Shot Learning from Demonstration of Contact-Rich Robotic Manipulation by Identifying Physical Interactions
  8. 2026ACDC: Adaptive Curriculum Planning with Dynamic Contrastive Control for Goal-Conditioned Reinforcement Learning in Robotic Manipulation