mlsanityData to 7 Oct 2026

Grounding Bodily Awareness in Visual Representations for Efficient Policy Learning

arXiv 2505.18487 ↗First posted 24 May 2025cs.RO cs.CV cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Learning effective visual representations for robotic manipulation remains a fundamental challenge due to the complex body dynamics involved in action execution. In this paper, we study how visual representations that carry body-relevant cues can enable efficient policy learning for downstream robotic manipulation tasks. We present Inter-token Contrast (ICon), a contrastive learning method applied to the token-level representations of Vision Transformers (ViTs). ICon enforces a separation in the feature space between agent-specific and environment-specific tokens, resulting in agent-centric visual representations that embed body-specific inductive biases. This framework can be seamlessly integrated into end-to-end policy learning by incorporating the contrastive loss as an auxiliary objective. Our experiments show that ICon not only improves policy performance across various manipulation tasks but also facilitates policy transfer across different robots. The project website: https://inter-token-contrast.github.io/icon/

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 41 of its 48 references that are in the map. First posted on arXiv 24 May 2025.

Builds on

41 of 48 references.

References · 41

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  6. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  7. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  8. 2023Segment Anythingcited by 8,158
  9. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  10. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  11. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  12. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
29 more
  1. 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,833
  2. 2024SAM 2: Segment Anything in Images and Videoscited by 2,784
  3. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
  4. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,490
  5. 2015End-to-End Training of Deep Visuomotor Policiescited by 847
  6. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 745
  7. 2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulationcited by 699
  8. 2022R3M: A Universal Visual Representation for Robot Manipulationcited by 678
  9. 2019RLBench: The Robot Learning Benchmark & Learning Environmentcited by 669
  10. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 573
  11. 2020robosuite: A Modular Simulation Framework and Benchmark for Robot Learningcited by 484
  12. 2017Time-Contrastive Networks: Self-Supervised Learning from Videocited by 453
  13. 2021Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learningcited by 359
  14. 2021Deep ViT Features as Dense Visual Descriptorscited by 327
  15. 2022Behavior Transformers: Cloning k modes with one stonecited by 299
  16. 2019Improving Sample Efficiency in Model-Free Reinforcement Learning from Imagescited by 268
  17. 2022Masked Visual Pre-training for Motor Controlcited by 246
  18. 2022Real-World Robot Learning with Masked Visual Pre-trainingcited by 241
  19. 2023Language-Driven Representation Learning for Roboticscited by 163
  20. 2024Behavior Generation with Latent Actionscited by 159
  21. 2020Transformers for One-Shot Visual Imitationcited by 77
  22. 2023XSkill: Cross Embodiment Skill Discoverycited by 74
  23. 2024In-Context Imitation Learning via Next-Token Predictioncited by 65
  24. 2023Crossway Diffusion: Improving Diffusion-based Visuomotor Policy via Self-supervised Learningcited by 37
  25. 2020Masked Contrastive Representation Learning for Reinforcement Learningcited by 32
  26. 2022ContrastMask: Contrastive Learning to Segment Every Thingcited by 19
  27. 2023Efficient RL via Disentangled Environment and Agent Representationscited by 11
  28. 2017Bodily aware soft robots: integration of proprioceptive and exteroceptive sensorscited by 5
  29. 2024DEAR: Disentangled Environment and Agent Representations for Reinforcement Learning without Reconstructioncited by 5

Built on by

Not cited yet.