# Large Scale Generative Multimodal Attribute Extraction for E-commerce Attributes

arXiv: https://arxiv.org/abs/2306.00379 · HTML: https://mlsanity.com/paper/2306.00379/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 14 references that are in the map.
- First posted on arXiv 1 Jun 2023.

## Builds on (most cited first, 10 of 10)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Xception: Deep Learning with Depthwise Separable Convolutions](https://mlsanity.com/paper/1610.02357/) (2016)
- [OpenTag: Open Attribute Value Extraction from Product Profiles [Deep Learning, Active Learning, Named Entity Recognition]](https://mlsanity.com/paper/1806.01264/) (2018)
- [Vision Guided Generative Pre-trained Language Models for Multimodal Abstractive Summarization](https://mlsanity.com/paper/2109.02401/) (2021)
- [PAM: Understanding Product Images in Cross Product Category Attribute Extraction](https://mlsanity.com/paper/2106.04630/) (2021)
- [CMA-CLIP: Cross-Modality Attention CLIP for Image-Text Classification](https://mlsanity.com/paper/2112.03562/) (2021)
- [LaTeX-Numeric: Language-agnostic Text attribute eXtraction for E-commerce Numeric Attributes](https://mlsanity.com/paper/2104.09576/) (2021)
- [NER-MQMRC: Formulating Named Entity Recognition as Multi Question Machine Reading Comprehension](https://mlsanity.com/paper/2205.05904/) (2022)

## Built on by (most cited first, 11 of 11)

- [ImplicitAVE: An Open-Source Dataset and Multimodal LLMs Benchmark for Implicit Attribute Value Extraction](https://mlsanity.com/paper/2404.15592/) (2024)
- [EIVEN: Efficient Implicit Attribute Value Extraction using Multimodal LLM](https://mlsanity.com/paper/2404.08886/) (2024)
- [TACLR: A Scalable and Efficient Retrieval-based Method for Industrial Product Attribute Value Identification](https://mlsanity.com/paper/2501.03835/) (2025)
- [MoEMoE: Question Guided Dense and Scalable Sparse Mixture-of-Expert for Multi-source Multi-modal Answering](https://mlsanity.com/paper/2503.06296/) (2025)
- [Mitigating Object Hallucinations in LVLMs via Attention Imbalance Rectification](https://mlsanity.com/paper/2603.24058/) (2026)
- [A Framework for Leveraging Partially-Labeled Data for Product Attribute-Value Identification](https://mlsanity.com/paper/2405.10918/) (2024)
- [JD Oxygen AI Item Center (Oxygen AIIC) V1: An Industrial-Scale LLM/VLM-Centric Solution for Item Understanding, Management, and Applications](https://mlsanity.com/paper/2606.28070/) (2026)
- [TRACE: Agentic Catalog Enrichment with Multi-source Evidence Grounding](https://mlsanity.com/paper/2608.20844/) (2026)
- [HMGCLIP: Heterogeneous Multi-Granularity Contrastive Learning for E-commerce Representation Learning](https://mlsanity.com/paper/2608.24467/) (2026)
- [CVE-SAI: Counterfactual Visual Evidence-Guided Selective Attribute Indexing for Risk-Controlled E-commerce Search](https://mlsanity.com/paper/2608.25023/) (2026)
- [Scaling E-Commerce Attribute Extraction with Parallel Decoding](https://mlsanity.com/paper/2609.09716/) (2026)

## Abstract (arXiv)

E-commerce websites (e.g. Amazon) have a plethora of structured and unstructured information (text and images) present on the product pages. Sellers often either don't label or mislabel values of the attributes (e.g. color, size etc.) for their products. Automatically identifying these attribute values from an eCommerce product page that contains both text and images is a challenging task, especially when the attribute value is not explicitly mentioned in the catalog. In this paper, we present a scalable solution for this problem where we pose attribute extraction problem as a question-answering task, which we solve using MXT, consisting of three key components: (i) MAG (Multimodal Adaptation Gate), (ii) Xception network, and (iii) T5 encoder-decoder. Our system consists of a generative model that generates attribute-values for a given product by using both textual and visual characteristics (e.g. images) of the product. We show that our system is capable of handling zero-shot attribute prediction (when attribute value is not seen in training data) and value-absent prediction (when attribute value is not mentioned in the text) which are missing in traditional classification-based and NER-based models respectively. We have trained our models using distant supervision, removing dependency on human labeling, thus making them practical for real-world applications. With this framework, we are able to train a single model for 1000s of (product-type, attribute) pairs, thus reducing the overhead of training and maintaining separate models. Extensive experiments on two real world datasets show that our framework improves the absolute recall@90P by 10.16% and 6.9% from the existing state of the art models. In a popular e-commerce store, we have deployed our models for 1000s of (product-type, attribute) pairs.