# Survey on Deep Multi-modal Data Analytics: Collaboration, Rivalry and Fusion

arXiv: https://arxiv.org/abs/2006.08159 · HTML: https://mlsanity.com/paper/2006.08159/

## Key facts

- Cited by 53 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Jun 2020.

## Built on by (most cited first, 30 of 53)

- [Generalizing Graph Neural Networks on Out-Of-Distribution Graphs](https://mlsanity.com/paper/2111.10657/) (2021)
- [Detecting and Grounding Multi-Modal Media Manipulation and Beyond](https://mlsanity.com/paper/2309.14203/) (2023)
- [Enhancing multimodal cooperation via sample-level modality valuation](https://mlsanity.com/paper/2309.06255/) (2023)
- [Self-Supervised Discriminative Feature Learning for Deep Multi-View Clustering](https://mlsanity.com/paper/2103.15069/) (2021)
- [Reinforced Neighborhood Selection Guided Multi-Relational Graph Neural Networks](https://mlsanity.com/paper/2104.07886/) (2021)
- [M3Net: Multi-view Encoding, Matching, and Fusion for Few-shot Fine-grained Action Recognition](https://mlsanity.com/paper/2308.03063/) (2023)
- [Structured Graph Learning for Scalable Subspace Clustering: From Single-view to Multi-view](https://mlsanity.com/paper/2102.07943/) (2021)
- [Switchable Online Knowledge Distillation](https://mlsanity.com/paper/2209.04996/) (2022)
- [Centering the Value of Every Modality: Towards Efficient and Resilient Modality-agnostic Semantic Segmentation](https://mlsanity.com/paper/2407.11344/) (2024)
- [Asymmetric Cross-Scale Alignment for Text-Based Person Search](https://mlsanity.com/paper/2212.11958/) (2022)
- [CLIP-Powered Domain Generalization and Domain Adaptation: A Comprehensive Survey](https://mlsanity.com/paper/2504.14280/) (2025)
- [Delving Globally into Texture and Structure for Image Inpainting](https://mlsanity.com/paper/2209.08217/) (2022)
- [Rethinking Data-Free Quantization as a Zero-Sum Game](https://mlsanity.com/paper/2302.09572/) (2023)
- [UniS-MMC: Multimodal Classification via Unimodality-supervised Multimodal Contrastive Learning](https://mlsanity.com/paper/2305.09299/) (2023)
- [MAGIC++: Efficient and Resilient Modality-Agnostic Semantic Segmentation via Hierarchical Modality Selection](https://mlsanity.com/paper/2412.16876/) (2024)
- [SHAPE: An Unified Approach to Evaluate the Contribution and Cooperation of Individual Modalities](https://mlsanity.com/paper/2205.00302/) (2022)
- [Multi-Modal Foundation Models for Computational Pathology: A Survey](https://mlsanity.com/paper/2503.09091/) (2025)
- [ChatGPT-Like Large-Scale Foundation Models for Prognostics and Health Management: A Survey and Roadmaps](https://mlsanity.com/paper/2305.06472/) (2023)
- [Multi-view MERA Subspace Clustering](https://mlsanity.com/paper/2305.09095/) (2023)
- [Scalable Incomplete Multi-View Clustering with Structure Alignment](https://mlsanity.com/paper/2308.16541/) (2023)
- [Learning Comprehensive Representations with Richer Self for Text-to-Image Person Re-Identification](https://mlsanity.com/paper/2310.11210/) (2023)
- [Masked Two-channel Decoupling Framework for Incomplete Multi-view Weak Multi-label Learning](https://mlsanity.com/paper/2404.17340/) (2024)
- [From CNNs to Transformers in Multimodal Human Action Recognition: A Survey](https://mlsanity.com/paper/2405.15813/) (2024)
- [Fine-grained Cross-modal Fusion based Refinement for Text-to-Image Synthesis](https://mlsanity.com/paper/2302.08706/) (2023)
- [Reachable Distance Function for KNN Classification](https://mlsanity.com/paper/2103.09704/) (2021)
- [PVSeRF: Joint Pixel-, Voxel- and Surface-Aligned Radiance Field for Single-Image Novel View Synthesis](https://mlsanity.com/paper/2202.04879/) (2022)
- [Efficient Multi-View Graph Clustering with Local and Global Structure Preservation](https://mlsanity.com/paper/2309.00024/) (2023)
- [Manifold-based Incomplete Multi-view Clustering via Bi-Consistency Guidance](https://mlsanity.com/paper/2405.10987/) (2024)
- [Finger Pose Estimation for Under-screen Fingerprint Sensor](https://mlsanity.com/paper/2505.02481/) (2025)
- [Reinforcement Learning based Path Exploration for Sequential Explainable Recommendation](https://mlsanity.com/paper/2111.12262/) (2021)

## Abstract (arXiv)

With the development of web technology, multi-modal or multi-view data has surged as a major stream for big data, where each modal/view encodes individual property of data objects. Often, different modalities are complementary to each other. Such fact motivated a lot of research attention on fusing the multi-modal feature spaces to comprehensively characterize the data objects. Most of the existing state-of-the-art focused on how to fuse the energy or information from multi-modal spaces to deliver a superior performance over their counterparts with single modal. Recently, deep neural networks have exhibited as a powerful architecture to well capture the nonlinear distribution of high-dimensional multimedia data, so naturally does for multi-modal data. Substantial empirical studies are carried out to demonstrate its advantages that are benefited from deep multi-modal methods, which can essentially deepen the fusion from multi-modal deep feature spaces. In this paper, we provide a substantial overview of the existing state-of-the-arts on the filed of multi-modal data analytics from shallow to deep spaces. Throughout this survey, we further indicate that the critical components for this field go to collaboration, adversarial competition and fusion over multi-modal spaces. Finally, we share our viewpoints regarding some future directions on this field.