# Mitigating Modality Bias in Multi-modal Entity Alignment from a Causal Perspective

arXiv: https://arxiv.org/abs/2504.19458 · HTML: https://mlsanity.com/paper/2504.19458/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 41 of its 67 references that are in the map.
- First posted on arXiv 28 Apr 2025.

## Builds on (most cited first, 30 of 41)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Shortcut Learning in Deep Neural Networks](https://mlsanity.com/paper/2004.07780/) (2020)
- [Diffusion Models: A Comprehensive Survey of Methods and Applications](https://mlsanity.com/paper/2209.00796/) (2022)
- [Counterfactual VQA: A Cause-Effect Look at Language Bias](https://mlsanity.com/paper/2006.04315/) (2020)
- [Multilingual Knowledge Graph Embeddings for Cross-lingual Knowledge Alignment](https://mlsanity.com/paper/1611.03954/) (2016)
- [Cross-lingual Entity Alignment via Joint Attribute-Preserving Embedding](https://mlsanity.com/paper/1708.05045/) (2017)
- [Relation-Aware Entity Alignment for Heterogeneous Knowledge Graphs](https://mlsanity.com/paper/1908.08210/) (2019)
- [MMKG: Multi-Modal Knowledge Graphs](https://mlsanity.com/paper/1903.05485/) (2019)
- [Knowledge Graph Alignment Network with Gated Multi-hop Neighborhood Aggregation](https://mlsanity.com/paper/1911.08936/) (2019)
- [Multi-Channel Graph Neural Network for Entity Alignment](https://mlsanity.com/paper/1908.09898/) (2019)
- [Multi-view Knowledge Graph Embedding for Entity Alignment](https://mlsanity.com/paper/1906.02390/) (2019)
- [Multi-Modal Knowledge Graph Construction and Application: A Survey](https://mlsanity.com/paper/2202.05786/) (2022)
- [Exploring and Evaluating Attributes, Values, and Structures for Entity Alignment](https://mlsanity.com/paper/2010.03249/) (2020)
- [Relational Reflection Entity Alignment](https://mlsanity.com/paper/2008.07962/) (2020)
- [Co-training Embeddings of Knowledge Graphs and Entity Descriptions for Cross-lingual Entity Alignment](https://mlsanity.com/paper/1806.06478/) (2018)
- [Visual Pivoting for (Unsupervised) Entity Alignment](https://mlsanity.com/paper/2009.13603/) (2020)
- [TransEdge: Translating Relation-contextualized Embeddings for Knowledge Graphs](https://mlsanity.com/paper/2004.13579/) (2020)
- [Multi-modal Contrastive Representation Learning for Entity Alignment](https://mlsanity.com/paper/2209.00891/) (2022)
- [Boosting the Speed of Entity Alignment 10*: Dual Attention Matching Network with Normalized Hard Sample Mining](https://mlsanity.com/paper/2103.15452/) (2021)
- [MEAformer: Multi-modal Entity Alignment Transformer for Meta Modality Hybrid](https://mlsanity.com/paper/2212.14454/) (2022)
- [Attribute-Consistent Knowledge Graph Representation Learning for Multi-Modal Entity Alignment](https://mlsanity.com/paper/2304.01563/) (2023)
- [Simple but Effective Raw-Data Level Multimodal Fusion for Composed Image Retrieval](https://mlsanity.com/paper/2404.15875/) (2024)
- [NativE: Multi-modal Knowledge Graph Completion in the Wild](https://mlsanity.com/paper/2406.17605/) (2024)
- [Rethinking Uncertainly Missing and Ambiguous Visual Modality in Multi-Modal Entity Alignment](https://mlsanity.com/paper/2307.16210/) (2023)
- [Multi-modal Entity Alignment in Hyperbolic Space](https://mlsanity.com/paper/2106.03619/) (2021)
- [AliMe MKG: A Multi-modal Knowledge Graph for Live-streaming E-commerce](https://mlsanity.com/paper/2109.07411/) (2021)
- [Revisit and Outstrip Entity Alignment: A Perspective of Generative Models](https://mlsanity.com/paper/2305.14651/) (2023)

## Built on by (most cited first, 1 of 1)

- [Panning for Gold: Expanding Domain-Specific Knowledge Graphs with General Knowledge](https://mlsanity.com/paper/2601.10485/) (2026)

## Abstract (arXiv)

Multi-Modal Entity Alignment (MMEA) aims to retrieve equivalent entities from different Multi-Modal Knowledge Graphs (MMKGs), a critical information retrieval task. Existing studies have explored various fusion paradigms and consistency constraints to improve the alignment of equivalent entities, while overlooking that the visual modality may not always contribute positively. Empirically, entities with low-similarity images usually generate unsatisfactory performance, highlighting the limitation of overly relying on visual features. We believe the model can be biased toward the visual modality, leading to a shortcut image-matching task. To address this, we propose a counterfactual debiasing framework for MMEA, termed CDMEA, which investigates visual modality bias from a causal perspective. Our approach aims to leverage both visual and graph modalities to enhance MMEA while suppressing the direct causal effect of the visual modality on model predictions. By estimating the Total Effect (TE) of both modalities and excluding the Natural Direct Effect (NDE) of the visual modality, we ensure that the model predicts based on the Total Indirect Effect (TIE), effectively utilizing both modalities and reducing visual modality bias. Extensive experiments on 9 benchmark datasets show that CDMEA outperforms 14 state-of-the-art methods, especially in low-similarity, high-noise, and low-resource data scenarios.