# Multimodal and Crossmodal AI for Smart Data Analysis

arXiv: https://arxiv.org/abs/2209.01308 · HTML: https://mlsanity.com/paper/2209.01308/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 26 references that are in the map.
- First posted on arXiv 3 Sep 2022.

## Builds on (most cited first, 7 of 7)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Multimodal Machine Learning: A Survey and Taxonomy](https://mlsanity.com/paper/1705.09406/) (2017)
- [A Comprehensive Survey on Cross-modal Retrieval](https://mlsanity.com/paper/1607.06215/) (2016)
- [Survey on Deep Multi-modal Data Analytics: Collaboration, Rivalry and Fusion](https://mlsanity.com/paper/2006.08159/) (2020)
- [Applications of deep learning in traffic congestion detection, prediction and alleviation: A survey](https://mlsanity.com/paper/2102.09759/) (2021)
- [Self-Supervised learning with cross-modal transformers for emotion recognition](https://mlsanity.com/paper/2011.10652/) (2020)
- [When Machine Learning Meets Congestion Control: A Survey and Comparison](https://mlsanity.com/paper/2010.11397/) (2020)

## Built on by (most cited first, 1 of 1)

- [Beyond Text: Implementing Multimodal Large Language Model-Powered Multi-Agent Systems Using a No-Code Platform](https://mlsanity.com/paper/2501.00750/) (2025)

## Abstract (arXiv)

Recently, the multimodal and crossmodal AI techniques have attracted the attention of communities. The former aims to collect disjointed and heterogeneous data to compensate for complementary information to enhance robust prediction. The latter targets to utilize one modality to predict another modality by discovering the common attention sharing between them. Although both approaches share the same target: generate smart data from collected raw data, the former demands more modalities while the latter aims to decrease the variety of modalities. This paper first discusses the role of multimodal and crossmodal AI in smart data analysis in general. Then, we introduce the multimodal and crossmodal AI framework (MMCRAI) to balance the abovementioned approaches and make it easy to scale into different domains. This framework is integrated into xDataPF (the cross-data platform https://www.xdata.nict.jp/). We also introduce and discuss various applications built on this framework and xDataPF.