# Is a picture worth a thousand words? A Deep Multi-Modal Fusion Architecture for Product Classification in e-commerce

arXiv: https://arxiv.org/abs/1611.09534 · HTML: https://mlsanity.com/paper/1611.09534/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Nov 2016.

## Built on by (most cited first, 7 of 7)

- [A Survey on Multi-modal Summarization](https://mlsanity.com/paper/2109.05199/) (2021)
- [Multimodal Hate Speech Detection from Bengali Memes and Texts](https://mlsanity.com/paper/2204.10196/) (2022)
- [Online Limited Memory Neural-Linear Bandits with Likelihood Matching](https://mlsanity.com/paper/2102.03799/) (2021)
- [Multi-Modal Attribute Extraction for E-Commerce](https://mlsanity.com/paper/2203.03441/) (2022)
- [EAML: Ensemble Self-Attention-based Mutual Learning Network for Document Image Classification](https://mlsanity.com/paper/2305.06923/) (2023)
- [Multimodal Approach for Harmonized System Code Prediction](https://mlsanity.com/paper/2406.04349/) (2024)
- [Cross-Platform E-Commerce Product Categorization and Recategorization: A Multimodal Hierarchical Classification Approach](https://mlsanity.com/paper/2508.20013/) (2025)

## Abstract (arXiv)

Classifying products into categories precisely and efficiently is a major challenge in modern e-commerce. The high traffic of new products uploaded daily and the dynamic nature of the categories raise the need for machine learning models that can reduce the cost and time of human editors. In this paper, we propose a decision level fusion approach for multi-modal product classification using text and image inputs. We train input specific state-of-the-art deep neural networks for each input source, show the potential of forging them together into a multi-modal architecture and train a novel policy network that learns to choose between them. Finally, we demonstrate that our multi-modal network improves the top-1 accuracy % over both networks on a real-world large-scale product classification dataset that we collected fromWalmart.com. While we focus on image-text fusion that characterizes e-commerce domains, our algorithms can be easily applied to other modalities such as audio, video, physical sensors, etc.