# Self-Supervised Learning-Based Multimodal Prediction on Prosocial Behavior Intentions

arXiv: https://arxiv.org/abs/2507.08238 · HTML: https://mlsanity.com/paper/2507.08238/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 25 references that are in the map.
- First posted on arXiv 11 Jul 2025.

## Builds on (most cited first, 9 of 9)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing](https://mlsanity.com/paper/2110.13900/) (2021)
- [Dawn of the transformer era in speech emotion recognition: closing the valence gap](https://mlsanity.com/paper/2203.07378/) (2022)
- [DeepTake: Prediction of Driver Takeover Behavior using Multimodal Data](https://mlsanity.com/paper/2012.15441/) (2020)
- [Beyond Empirical Windowing: An Attention-Based Approach for Trust Prediction in Autonomous Vehicles](https://mlsanity.com/paper/2312.10209/) (2023)
- [Should I Help a Delivery Robot? Cultivating Prosocial Norms through Observations](https://mlsanity.com/paper/2403.19027/) (2024)
- [DualTake: Predicting Takeovers across Mobilities for Future Personalized Mobility Services](https://mlsanity.com/paper/2401.10175/) (2024)
- [How is the Pilot Doing: VTOL Pilot Workload Estimation by Multimodal Machine Learning on Psycho-physiological Signals](https://mlsanity.com/paper/2406.06448/) (2024)

## Abstract (arXiv)

Human state detection and behavior prediction have seen significant advancements with the rise of machine learning and multimodal sensing technologies. However, predicting prosocial behavior intentions in mobility scenarios, such as helping others on the road, is an underexplored area. Current research faces a major limitation. There are no large, labeled datasets available for prosocial behavior, and small-scale datasets make it difficult to train deep-learning models effectively. To overcome this, we propose a self-supervised learning approach that harnesses multi-modal data from existing physiological and behavioral datasets. By pre-training our model on diverse tasks and fine-tuning it with a smaller, manually labeled prosocial behavior dataset, we significantly enhance its performance. This method addresses the data scarcity issue, providing a more effective benchmark for prosocial behavior prediction, and offering valuable insights for improving intelligent vehicle systems and human-machine interaction.