# Multimodal Attribute Extraction

arXiv: https://arxiv.org/abs/1711.11118 · HTML: https://mlsanity.com/paper/1711.11118/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Nov 2017.

## Built on by (most cited first, 7 of 7)

- [MAVE: A Product Dataset for Multi-source Attribute Value Extraction](https://mlsanity.com/paper/2112.08663/) (2021)
- [K-PLUG: Knowledge-injected Pre-trained Language Model for Natural Language Understanding and Generation in E-Commerce](https://mlsanity.com/paper/2104.06960/) (2021)
- [PAE: LLM-based Product Attribute Extraction for E-Commerce Fashion Trends](https://mlsanity.com/paper/2405.17533/) (2024)
- [Exploring Generative Models for Joint Attribute Value Extraction from Product Titles](https://mlsanity.com/paper/2208.07130/) (2022)
- [Boosting Multi-Modal E-commerce Attribute Value Extraction via Unified Learning Scheme and Dynamic Range Minimization](https://mlsanity.com/paper/2207.07278/) (2022)
- [ImPaKT: A Dataset for Open-Schema Knowledge Base Construction](https://mlsanity.com/paper/2212.10770/) (2022)
- [VideoAVE: A Multi-Attribute Video-to-Text Attribute Value Extraction Dataset and Benchmark Models](https://mlsanity.com/paper/2508.11801/) (2025)

## Abstract (arXiv)

The broad goal of information extraction is to derive structured information from unstructured data. However, most existing methods focus solely on text, ignoring other types of unstructured data such as images, video and audio which comprise an increasing portion of the information on the web. To address this shortcoming, we propose the task of multimodal attribute extraction. Given a collection of unstructured and semi-structured contextual information about an entity (such as a textual description, or visual depictions) the task is to extract the entity's underlying attributes. In this paper, we provide a dataset containing mixed-media data for over 2 million product items along with 7 million attribute-value pairs describing the items which can be used to train attribute extractors in a weakly supervised manner. We provide a variety of baselines which demonstrate the relative effectiveness of the individual modes of information towards solving the task, as well as study human performance.