# MoralCLIP: Contrastive Alignment of Vision-and-Language Representations with Moral Foundations Theory

arXiv: https://arxiv.org/abs/2506.05696 · HTML: https://mlsanity.com/paper/2506.05696/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 28 of its 74 references that are in the map.
- First posted on arXiv 6 Jun 2025.

## Placed in

- Multimodal LLMs / vision-language models / [VLM benchmarks and visual reasoning](https://mlsanity.com/mllm/vlm-benchmarks-and-visual-reasoning/)

## Builds on (most cited first, 28 of 28)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision](https://mlsanity.com/paper/2102.05918/) (2021)
- [Sigmoid Loss for Language Image Pre-Training](https://mlsanity.com/paper/2303.15343/) (2023)
- [InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning](https://mlsanity.com/paper/2305.06500/) (2023)
- [LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairs](https://mlsanity.com/paper/2111.02114/) (2021)
- [Aligning AI With Shared Human Values](https://mlsanity.com/paper/2008.02275/) (2020)
- [PaliGemma: A versatile 3B VLM for transfer](https://mlsanity.com/paper/2407.07726/) (2024)
- [Long-CLIP: Unlocking the Long-Text Capability of CLIP](https://mlsanity.com/paper/2403.15378/) (2024)
- [Sampling Matters in Deep Embedding Learning](https://mlsanity.com/paper/1706.07567/) (2017)
- [AnomalyCLIP: Object-agnostic Prompt Learning for Zero-shot Anomaly Detection](https://mlsanity.com/paper/2310.18961/) (2023)
- [DeAR: Debiasing Vision-Language Models with Additive Residuals](https://mlsanity.com/paper/2303.10431/) (2023)
- [Image Retrieval from Contextual Descriptions](https://mlsanity.com/paper/2203.15867/) (2022)
- [MultiModal Bias: Introducing a Framework for Stereotypical Bias Assessment beyond Gender and Race in Vision Language Models](https://mlsanity.com/paper/2303.12734/) (2023)
- [MoralBERT: A Fine-Tuned Language Model for Capturing Moral Values in Social Discussions](https://mlsanity.com/paper/2403.07678/) (2024)
- [A Data Fusion Framework for Multi-Domain Morality Learning](https://mlsanity.com/paper/2304.02144/) (2023)
- [Measuring Moral Dimensions in Social Media with Mformer](https://mlsanity.com/paper/2311.10219/) (2023)
- [Is CLIP the main roadblock for fine-grained open-world perception?](https://mlsanity.com/paper/2404.03539/) (2024)
- [VIVA: A Benchmark for Vision-Grounded Decision-Making with Human Values](https://mlsanity.com/paper/2407.03000/) (2024)
- [Exploring How Generative MLLMs Perceive More Than CLIP with the Same Vision Encoder](https://mlsanity.com/paper/2411.05195/) (2024)
- [MM-MoralBench: A MultiModal Moral Evaluation Benchmark for Large Vision-Language Models](https://mlsanity.com/paper/2412.20718/) (2024)
- [MOKA: Moral Knowledge Augmentation for Moral Event Extraction](https://mlsanity.com/paper/2311.09733/) (2023)
- [Development and Validation of the Japanese Moral Foundations Dictionary](https://mlsanity.com/paper/1804.00871/) (2018)
- [Blind Dates: Examining the Expression of Temporality in Historical Photographs](https://mlsanity.com/paper/2310.06633/) (2023)
- [Zero-shot Visual Commonsense Immorality Prediction](https://mlsanity.com/paper/2211.05521/) (2022)
- [E2MoCase: A Dataset for Emotional, Event and Moral Observations in News Articles on High-impact Legal Cases](https://mlsanity.com/paper/2409.09001/) (2024)

## Abstract (arXiv)

Recent advances in vision-language models have enabled rich semantic understanding across modalities. However, these encoding methods lack the ability to interpret or reason about the moral dimensions of content-a crucial aspect of human cognition. In this paper, we address this gap by introducing MoralCLIP, a novel embedding representation method that extends multimodal learning with explicit moral grounding based on Moral Foundations Theory (MFT). Our approach integrates visual and textual moral cues into a unified embedding space, enabling cross-modal moral alignment. MoralCLIP is grounded on the multi-label dataset Social-Moral Image Database to identify co-occurring moral foundations in visual content. For MoralCLIP training, we design a moral data augmentation strategy to scale our annotated dataset to 15,000 image-text pairs labeled with MFT-aligned dimensions. Our results demonstrate that explicit moral supervision improves both unimodal and multimodal understanding of moral content, establishing a foundation for morally-aware AI systems capable of recognizing and aligning with human moral values.