# Large Scale Multimodal Classification Using an Ensemble of Transformer Models and Co-Attention

arXiv: https://arxiv.org/abs/2011.11735 · HTML: https://mlsanity.com/paper/2011.11735/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Nov 2020.

## Built on by (most cited first, 2 of 2)

- [Multimodal E-Commerce Product Classification Using Hierarchical Fusion](https://mlsanity.com/paper/2207.03305/) (2022)
- [MMRNet: Improving Reliability for Multimodal Object Detection and Segmentation for Bin Picking via Multimodal Redundancy](https://mlsanity.com/paper/2210.10842/) (2022)

## Abstract (arXiv)

Accurate and efficient product classification is significant for E-commerce applications, as it enables various downstream tasks such as recommendation, retrieval, and pricing. Items often contain textual and visual information, and utilizing both modalities usually outperforms classification utilizing either mode alone. In this paper we describe our methodology and results for the SIGIR eCom Rakuten Data Challenge. We employ a dual attention technique to model image-text relationships using pretrained language and image embeddings. While dual attention has been widely used for Visual Question Answering(VQA) tasks, ours is the first attempt to apply the concept for multimodal classification.