# M2CURL: Sample-Efficient Multimodal Reinforcement Learning via Self-Supervised Representation Learning for Robotic Manipulation

arXiv: https://arxiv.org/abs/2401.17032 · HTML: https://mlsanity.com/paper/2401.17032/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods.
- Builds on 29 of its 49 references that are in the map.
- First posted on arXiv 30 Jan 2024.

## Placed in

- Deep reinforcement learning methods / [RL for robot locomotion and manipulation](https://mlsanity.com/deeprl/rl-for-robot-locomotion-and-manipulation/)

## Builds on (most cited first, 29 of 29)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Multimodal Machine Learning: A Survey and Taxonomy](https://mlsanity.com/paper/1705.09406/) (2017)
- [CURL: Contrastive Unsupervised Representations for Reinforcement Learning](https://mlsanity.com/paper/2004.04136/) (2020)
- [Multimodal Deep Learning](https://mlsanity.com/paper/2301.04856/) (2023)
- [More Than a Feeling: Learning to Grasp and Regrasp using Vision and Touch](https://mlsanity.com/paper/1805.11085/) (2018)
- [Touch and Go: Learning from Human-Collected Vision and Touch](https://mlsanity.com/paper/2211.12498/) (2022)
- [Making Sense of Vision and Touch: Learning Multimodal Representations for Contact-Rich Tasks](https://mlsanity.com/paper/1907.13098/) (2019)
- [See, Hear, and Feel: Smart Sensory Fusion for Robotic Manipulation](https://mlsanity.com/paper/2212.03858/) (2022)
- [The Feeling of Success: Does Touch Sensing Help Predict Grasp Outcomes?](https://mlsanity.com/paper/1710.05512/) (2017)
- [Connecting Touch and Vision via Cross-Modal Prediction](https://mlsanity.com/paper/1906.06322/) (2019)
- [Dexterity from Touch: Self-Supervised Pre-Training of Tactile Representations with Robotic Play](https://mlsanity.com/paper/2303.12076/) (2023)
- [Visuo-Tactile Transformers for Manipulation](https://mlsanity.com/paper/2210.00121/) (2022)
- [Self-Supervised Visuo-Tactile Pretraining to Locate and Follow Garment Features](https://mlsanity.com/paper/2209.13042/) (2022)
- [Manipulation by Feel: Touch-Based Control with Deep Predictive Models](https://mlsanity.com/paper/1903.04128/) (2019)
- [Multimodal Visual-Tactile Representation Learning through Self-Supervised Contrastive Pre-Training](https://mlsanity.com/paper/2401.12024/) (2024)
- [ViTac: Feature Sharing between Vision and Tactile Sensing for Cloth Texture Recognition](https://mlsanity.com/paper/1802.07490/) (2018)
- [Generating Visual Scenes from Touch](https://mlsanity.com/paper/2309.15117/) (2023)
- [Tactile Dexterity: Manipulation Primitives with Tactile Feedback](https://mlsanity.com/paper/2002.03236/) (2020)
- [EmoNets: Multimodal deep learning approaches for emotion recognition in video](https://mlsanity.com/paper/1503.01800/) (2015)
- [Tactile Gym 2.0: Sim-to-real Deep Reinforcement Learning for Comparing Low-cost High-Resolution Robot Touch](https://mlsanity.com/paper/2207.10763/) (2022)
- [Learning to Identify Object Instances by Touch: Tactile Recognition via Multimodal Matching](https://mlsanity.com/paper/1903.03591/) (2019)
- [Visuo-Haptic Object Perception for Robots: An Overview](https://mlsanity.com/paper/2203.11544/) (2022)
- [Multimodal Hierarchical Reinforcement Learning Policy for Task-Oriented Visual Dialog](https://mlsanity.com/paper/1805.03257/) (2018)
- [Multimodal Machine Translation with Reinforcement Learning](https://mlsanity.com/paper/1805.02356/) (2018)
- [Attention for Robot Touch: Tactile Saliency Prediction for Robust Sim-to-Real Tactile Control](https://mlsanity.com/paper/2307.14510/) (2023)

## Built on by (most cited first, 8 of 8)

- [Survey on Large Language Model-Enhanced Reinforcement Learning: Concept, Taxonomy, and Methods](https://mlsanity.com/paper/2404.00282/) (2024)
- [Binding Touch to Everything: Learning Unified Multimodal Tactile Representations](https://mlsanity.com/paper/2401.18084/) (2024)
- [ViTacGen: Robotic Pushing with Vision-to-Touch Generation](https://mlsanity.com/paper/2510.14117/) (2025)
- [Self-Supervised Multisensory Pretraining for Contact-Rich Robot Reinforcement Learning](https://mlsanity.com/paper/2511.14427/) (2025)
- [RA-Touch: Retrieval-Augmented Touch Understanding with Enriched Visual Data](https://mlsanity.com/paper/2505.14270/) (2025)
- [ViTaPEs: Visuotactile Position Encodings for Cross-Modal Alignment in Multimodal Transformers](https://mlsanity.com/paper/2505.20032/) (2025)
- [Human Centric General Physical Intelligence for Agile Manufacturing Automation](https://mlsanity.com/paper/2508.11960/) (2025)
- [VQ-Touch: A Data-Efficient Tactile Generation Framework Across Sensors and Scenarios](https://mlsanity.com/paper/2607.14728/) (2026)

## Abstract (arXiv)

One of the most critical aspects of multimodal Reinforcement Learning (RL) is the effective integration of different observation modalities. Having robust and accurate representations derived from these modalities is key to enhancing the robustness and sample efficiency of RL algorithms. However, learning representations in RL settings for visuotactile data poses significant challenges, particularly due to the high dimensionality of the data and the complexity involved in correlating visual and tactile inputs with the dynamic environment and task objectives. To address these challenges, we propose Multimodal Contrastive Unsupervised Reinforcement Learning (M2CURL). Our approach employs a novel multimodal self-supervised learning technique that learns efficient representations and contributes to faster convergence of RL algorithms. Our method is agnostic to the RL algorithm, thus enabling its integration with any available RL algorithm. We evaluate M2CURL on the Tactile Gym 2 simulator and we show that it significantly enhances the learning efficiency in different manipulation tasks. This is evidenced by faster convergence rates and higher cumulative rewards per episode, compared to standard RL algorithms without our representation learning approach.