HMGCLIP: Heterogeneous Multi-Granularity Contrastive Learning for E-commerce Representation Learning
On the map: Multimodal LLMs / vision-language models → Document understanding and multimodal RAG
Abstract (arXiv)
Although recent Multimodal Large Language Models (MLLMs) have advanced general product understanding, they implicitly encode product information into global embeddings, thereby limiting their ability to capture fine-grained attributes. This limitation hinders performance in tasks requiring precise attribute discrimination, such as distinguishing subtle material differences among visually similar products. To address this challenge, we propose HMGCLIP, a unified multimodal embedding framework. By constructing a heterogeneous hypergraph, we leverage hypergraph topology to mine structure-aware hard negatives and align multi-granular semantics at both relation and hyperedge levels. This design enables a dual-granularity inference mechanism that dynamically fuses attribute evidence for both fine-grained and coarse-grained downstream tasks. Furthermore, we release a comprehensive fine-grained e-commerce dataset to facilitate future benchmarking. Extensive experiments on this new dataset and the public MAVE benchmark show that HMGCLIP outperforms strong multimodal encoders, MLLMs, and e-commerce baselines, validating the superiority of HMGCLIP.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Document understanding and multimodal RAG, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 20 of its 27 references that are in the map. First posted on arXiv 25 Aug 2026.
Builds on
20 of 27 references.
Closest ancestors · 1
Same direction, newest first
Other references · 19
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
- 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,864
- 2019Heterogeneous Graph Attention Networkcited by 734
- 2020Prototypical Contrastive Learning of Unsupervised Representationscited by 640
- 2020Contrastive Learning with Hard Negative Samplescited by 628
- 2020Graph Contrastive Learning with Adaptive Augmentationcited by 519
- 2026Qwen3-VL-Embedding and Qwen3-VL-Reranker: A Unified Framework for State-of-the-Art Multimodal Retrieval and Rankingcited by 230
- 2021UniGNN: a Unified Framework for Graph and Hypergraph Neural Networkscited by 130
- 2024NV-Retriever: Improving text embedding models with effective hard-negative miningcited by 84
- 2021MAVE: A Product Dataset for Multi-source Attribute Value Extractioncited by 53
- 2022Contrastive language and vision learning of general fashion conceptscited by 42
7 more
- 2021M5Product: Self-harmonized Contrastive Learning for E-commercial Multi-modal Pretrainingcited by 26
- 2023ExtractGPT: Exploring the Potential of Large Language Models for Product Attribute Value Extractioncited by 20
- 2021Knowledge Perceived Multi-modal Pretraining in E-commercecited by 18
- 2023Large Scale Generative Multimodal Attribute Extraction for E-commerce Attributescited by 11
- 2023Learning Instance-Level Representation for Large-Scale Multi-Modal Pretraining in E-commercecited by 8
- 2025HierPromptLM: A Pure PLM-based Framework for Representation Learning on Heterogeneous Text-rich Networkscited by 2
- 2026AutoPKG: An Automated Framework for Dynamic E-commerce Product-Attribute Knowledge Graph Constructioncited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026MOON3.0: Reasoning-aware Multimodal Representation Learning for E-commerce Product Understanding
- 2026AFMRL: Attribute-Enhanced Fine-Grained Multi-Modal Representation Learning in E-commerce
- 2025MOON2.0: Dynamic Modality-balanced Multimodal Representation Learning for E-commerce Product Understanding
- 2025MOON: Generative MLLM-based Multimodal Representation Learning for E-commerce Product Understanding
- 2026Adapting Vision-Language Models for E-commerce Understanding at Scale
- 2025Factorized Transport Alignment for Multimodal and Multiview E-commerce Representation Learning ↗
- 2025Compass-Embedding v4: Robust Contrastive Learning for Multilingual E-commerce Embeddings
- 2023PUMGPT: A Large Vision-Language Model for Product Understanding