# Multimodal Joint Attribute Prediction and Value Extraction for E-commerce Product

arXiv: https://arxiv.org/abs/2009.07162 · HTML: https://mlsanity.com/paper/2009.07162/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Sep 2020.

## Built on by (most cited first, 30 of 39)

- [Multi-Modal Knowledge Graph Construction and Application: A Survey](https://mlsanity.com/paper/2202.05786/) (2022)
- [MAVE: A Product Dataset for Multi-source Attribute Value Extraction](https://mlsanity.com/paper/2112.08663/) (2021)
- [EcomGPT: Instruction-tuning Large Language Models with Chain-of-Task Tasks for E-commerce](https://mlsanity.com/paper/2308.06966/) (2023)
- [AdaTag: Multi-Attribute Value Extraction from Product Profiles with Adaptive Decoding](https://mlsanity.com/paper/2106.02318/) (2021)
- [ExtractGPT: Exploring the Potential of Large Language Models for Product Attribute Value Extraction](https://mlsanity.com/paper/2310.12537/) (2023)
- [Jellyfish: A Large Language Model for Data Preprocessing](https://mlsanity.com/paper/2312.01678/) (2023)
- [PAM: Understanding Product Images in Cross Product Category Attribute Extraction](https://mlsanity.com/paper/2106.04630/) (2021)
- [ImplicitAVE: An Open-Source Dataset and Multimodal LLMs Benchmark for Implicit Attribute Value Extraction](https://mlsanity.com/paper/2404.15592/) (2024)
- [K-PLUG: Knowledge-injected Pre-trained Language Model for Natural Language Understanding and Generation in E-Commerce](https://mlsanity.com/paper/2104.06960/) (2021)
- [Simple and Effective Knowledge-Driven Query Expansion for QA-Based Product Attribute Extraction](https://mlsanity.com/paper/2206.14264/) (2022)
- [EIVEN: Efficient Implicit Attribute Value Extraction using Multimodal LLM](https://mlsanity.com/paper/2404.08886/) (2024)
- [Product Question Answering in E-Commerce: A Survey](https://mlsanity.com/paper/2302.08092/) (2023)
- [EcomGPT-CT: Continual Pre-training of E-commerce Large Language Models with Semi-structured Data](https://mlsanity.com/paper/2312.15696/) (2023)
- [AMELI: Enhancing Multimodal Entity Linking with Fine-Grained Attributes](https://mlsanity.com/paper/2305.14725/) (2023)
- [Structure Guided Multi-modal Pre-trained Transformer for Knowledge Graph Reasoning](https://mlsanity.com/paper/2307.03591/) (2023)
- [Grounding and Distinguishing Conceptual Vocabulary Through Similarity Learning in Embodied Simulations](https://mlsanity.com/paper/2305.13668/) (2023)
- [PUMGPT: A Large Vision-Language Model for Product Understanding](https://mlsanity.com/paper/2308.09568/) (2023)
- [A Survey of Ontology Expansion for Conversational Understanding](https://mlsanity.com/paper/2410.15019/) (2024)
- [Captions Speak Louder than Images: Generalizing Foundation Models for E-commerce from High-quality Multimodal Instruction Data](https://mlsanity.com/paper/2410.17337/) (2024)
- [TACLR: A Scalable and Efficient Retrieval-based Method for Industrial Product Attribute Value Identification](https://mlsanity.com/paper/2501.03835/) (2025)
- [Multi-Modal Attribute Extraction for E-Commerce](https://mlsanity.com/paper/2203.03441/) (2022)
- [PAE: LLM-based Product Attribute Extraction for E-Commerce Fashion Trends](https://mlsanity.com/paper/2405.17533/) (2024)
- [EAVE: Efficient Product Attribute Value Extraction via Lightweight Sparse-layer Interaction](https://mlsanity.com/paper/2406.06839/) (2024)
- [A Survey on Knowledge-Enhanced Pre-trained Language Models](https://mlsanity.com/paper/2212.13428/) (2022)
- [Product Information Extraction using ChatGPT](https://mlsanity.com/paper/2306.14921/) (2023)
- [Knowledge-Enhanced Multi-Label Few-Shot Product Attribute-Value Extraction](https://mlsanity.com/paper/2308.08413/) (2023)
- [Exploring Large Language Models for Product Attribute Value Identification](https://mlsanity.com/paper/2409.12695/) (2024)
- [IndustryBench-MIPU: Benchmarking Multi-Image Attribute Value Extraction for Industrial Products](https://mlsanity.com/paper/2606.14383/) (2026)
- [Exploring Generative Models for Joint Attribute Value Extraction from Product Titles](https://mlsanity.com/paper/2208.07130/) (2022)
- [PV2TEA: Patching Visual Modality to Textual-Established Information Extraction](https://mlsanity.com/paper/2306.01016/) (2023)

## Abstract (arXiv)

Product attribute values are essential in many e-commerce scenarios, such as customer service robots, product recommendations, and product retrieval. While in the real world, the attribute values of a product are usually incomplete and vary over time, which greatly hinders the practical applications. In this paper, we propose a multimodal method to jointly predict product attributes and extract values from textual product descriptions with the help of the product images. We argue that product attributes and values are highly correlated, e.g., it will be easier to extract the values on condition that the product attributes are given. Thus, we jointly model the attribute prediction and value extraction tasks from multiple aspects towards the interactions between attributes and values. Moreover, product images have distinct effects on our tasks for different product attributes and values. Thus, we selectively draw useful visual information from product images to enhance our model. We annotate a multimodal product attribute value dataset that contains 87,194 instances, and the experimental results on this dataset demonstrate that explicitly modeling the relationship between attributes and values facilitates our method to establish the correspondence between them, and selectively utilizing visual product information is necessary for the task. Our code and dataset will be released to the public.