# AnANet: Modeling Association and Alignment for Cross-modal Correlation Classification

arXiv: https://arxiv.org/abs/2109.00693 · HTML: https://mlsanity.com/paper/2109.00693/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 28 references that are in the map.
- First posted on arXiv 2 Sep 2021.

## Builds on (most cited first, 10 of 10)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Tensor Fusion Network for Multimodal Sentiment Analysis](https://mlsanity.com/paper/1707.07250/) (2017)
- [Stacked Cross Attention for Image-Text Matching](https://mlsanity.com/paper/1803.08024/) (2018)
- [Integrating Text and Image: Determining Multimodal Document Intent in Instagram Posts](https://mlsanity.com/paper/1904.09073/) (2019)
- [Clue: Cross-modal Coherence Modeling for Caption Generation](https://mlsanity.com/paper/2005.00908/) (2020)
- [Equal But Not The Same: Understanding the Implicit Relationship Between Persuasive Images and Text](https://mlsanity.com/paper/1807.08205/) (2018)
- [CITE: A Corpus of Image-Text Discourse Relations](https://mlsanity.com/paper/1904.06286/) (2019)
- [Image Specificity](https://mlsanity.com/paper/1502.04569/) (2015)
- [Multimodal Classification of Events in Social Media](https://mlsanity.com/paper/1601.00599/) (2016)

## Abstract (arXiv)

The explosive increase of multimodal data makes a great demand in many cross-modal applications that follow the strict prior related assumption. Thus researchers study the definition of cross-modal correlation category and construct various classification systems and predictive models. However, those systems pay more attention to the fine-grained relevant types of cross-modal correlation, ignoring lots of implicit relevant data which are often divided into irrelevant types. What's worse is that none of previous predictive models manifest the essence of cross-modal correlation according to their definition at the modeling stage. In this paper, we present a comprehensive analysis of the image-text correlation and redefine a new classification system based on implicit association and explicit alignment. To predict the type of image-text correlation, we propose the Association and Alignment Network according to our proposed definition (namely AnANet) which implicitly represents the global discrepancy and commonality between image and text and explicitly captures the cross-modal local relevance. The experimental results on our constructed new image-text correlation dataset show the effectiveness of our model.