# Hierarchical Knowledge Graph Construction from Images for Scalable E-Commerce

arXiv: https://arxiv.org/abs/2410.21237 · HTML: https://mlsanity.com/paper/2410.21237/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 39 references that are in the map.
- First posted on arXiv 28 Oct 2024.

## Builds on (most cited first, 19 of 19)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks](https://mlsanity.com/paper/2312.14238/) (2023)
- [How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suites](https://mlsanity.com/paper/2404.16821/) (2024)
- [Complex Embeddings for Simple Link Prediction](https://mlsanity.com/paper/1606.06357/) (2016)
- [A Survey on Knowledge Graph-Based Recommender Systems](https://mlsanity.com/paper/2003.00911/) (2020)
- [Knowledge Graph Prompting for Multi-Document Question Answering](https://mlsanity.com/paper/2308.11730/) (2023)
- [Extract, Define, Canonicalize: An LLM-based Framework for Knowledge Graph Construction](https://mlsanity.com/paper/2404.03868/) (2024)
- [RPC: A Large-Scale Retail Product Checkout Dataset](https://mlsanity.com/paper/1901.07249/) (2019)
- [RP2K: A Large-Scale Retail Product Dataset for Fine-Grained Image Classification](https://mlsanity.com/paper/2006.12634/) (2020)
- [Knowledge Graph Completion Models are Few-shot Learners: An Empirical Study of Relation Labeling in E-commerce with LLMs](https://mlsanity.com/paper/2305.09858/) (2023)
- [Research Commentary on Recommendations with Side Information: A Survey and Research Directions](https://mlsanity.com/paper/1909.12807/) (2019)
- [FolkScope: Intention Knowledge Graph Construction for E-commerce Commonsense Discovery](https://mlsanity.com/paper/2211.08316/) (2022)
- [Automated Construction of Theme-specific Knowledge Graphs](https://mlsanity.com/paper/2404.19146/) (2024)
- [Automatic Knowledge Graph Construction for Judicial Cases](https://mlsanity.com/paper/2404.09416/) (2024)
- [AliMe KBQA: Question Answering over Structured Knowledge for E-commerce Customer Service](https://mlsanity.com/paper/1912.05728/) (2019)

## Abstract (arXiv)

Knowledge Graph (KG) is playing an increasingly important role in various AI systems. For e-commerce, an efficient and low-cost automated knowledge graph construction method is the foundation of enabling various successful downstream applications. In this paper, we propose a novel method for constructing structured product knowledge graphs from raw product images. The method cooperatively leverages recent advances in the vision-language model (VLM) and large language model (LLM), fully automating the process and allowing timely graph updates. We also present a human-annotated e-commerce product dataset for benchmarking product property extraction in knowledge graph construction. Our method outperforms our baseline in all metrics and evaluated properties, demonstrating its effectiveness and bright usage potential.