# HMGCLIP: Heterogeneous Multi-Granularity Contrastive Learning for E-commerce Representation Learning

arXiv: https://arxiv.org/abs/2608.24467 · HTML: https://mlsanity.com/paper/2608.24467/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Document understanding and multimodal RAG, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 20 of its 27 references that are in the map.
- First posted on arXiv 25 Aug 2026.

## Placed in

- Multimodal LLMs / vision-language models / [Document understanding and multimodal RAG](https://mlsanity.com/mllm/document-understanding-and-multimodal-rag/)

## Builds on (most cited first, 20 of 20)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision](https://mlsanity.com/paper/2102.05918/) (2021)
- [Heterogeneous Graph Attention Network](https://mlsanity.com/paper/1903.07293/) (2019)
- [Prototypical Contrastive Learning of Unsupervised Representations](https://mlsanity.com/paper/2005.04966/) (2020)
- [Contrastive Learning with Hard Negative Samples](https://mlsanity.com/paper/2010.04592/) (2020)
- [Graph Contrastive Learning with Adaptive Augmentation](https://mlsanity.com/paper/2010.14945/) (2020)
- [Qwen3-VL-Embedding and Qwen3-VL-Reranker: A Unified Framework for State-of-the-Art Multimodal Retrieval and Ranking](https://mlsanity.com/paper/2601.04720/) (2026)
- [MM-Embed: Universal Multimodal Retrieval with Multimodal LLMs](https://mlsanity.com/paper/2411.02571/) (2024)
- [UniGNN: a Unified Framework for Graph and Hypergraph Neural Networks](https://mlsanity.com/paper/2105.00956/) (2021)
- [NV-Retriever: Improving text embedding models with effective hard-negative mining](https://mlsanity.com/paper/2407.15831/) (2024)
- [MAVE: A Product Dataset for Multi-source Attribute Value Extraction](https://mlsanity.com/paper/2112.08663/) (2021)
- [Contrastive language and vision learning of general fashion concepts](https://mlsanity.com/paper/2204.03972/) (2022)
- [M5Product: Self-harmonized Contrastive Learning for E-commercial Multi-modal Pretraining](https://mlsanity.com/paper/2109.04275/) (2021)
- [ExtractGPT: Exploring the Potential of Large Language Models for Product Attribute Value Extraction](https://mlsanity.com/paper/2310.12537/) (2023)
- [Knowledge Perceived Multi-modal Pretraining in E-commerce](https://mlsanity.com/paper/2109.00895/) (2021)
- [Large Scale Generative Multimodal Attribute Extraction for E-commerce Attributes](https://mlsanity.com/paper/2306.00379/) (2023)
- [Learning Instance-Level Representation for Large-Scale Multi-Modal Pretraining in E-commerce](https://mlsanity.com/paper/2304.02853/) (2023)
- [HierPromptLM: A Pure PLM-based Framework for Representation Learning on Heterogeneous Text-rich Networks](https://mlsanity.com/paper/2501.12857/) (2025)
- [AutoPKG: An Automated Framework for Dynamic E-commerce Product-Attribute Knowledge Graph Construction](https://mlsanity.com/paper/2604.16950/) (2026)

## Abstract (arXiv)

Although recent Multimodal Large Language Models (MLLMs) have advanced general product understanding, they implicitly encode product information into global embeddings, thereby limiting their ability to capture fine-grained attributes. This limitation hinders performance in tasks requiring precise attribute discrimination, such as distinguishing subtle material differences among visually similar products. To address this challenge, we propose HMGCLIP, a unified multimodal embedding framework. By constructing a heterogeneous hypergraph, we leverage hypergraph topology to mine structure-aware hard negatives and align multi-granular semantics at both relation and hyperedge levels. This design enables a dual-granularity inference mechanism that dynamically fuses attribute evidence for both fine-grained and coarse-grained downstream tasks. Furthermore, we release a comprehensive fine-grained e-commerce dataset to facilitate future benchmarking. Extensive experiments on this new dataset and the public MAVE benchmark show that HMGCLIP outperforms strong multimodal encoders, MLLMs, and e-commerce baselines, validating the superiority of HMGCLIP.