# Benchmarking Multimodal Large Language Models for Missing Modality Completion in Product Catalogues

arXiv: https://arxiv.org/abs/2601.19750 · HTML: https://mlsanity.com/paper/2601.19750/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Document understanding and multimodal RAG, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 29 of its 47 references that are in the map.
- First posted on arXiv 27 Jan 2026.

## Placed in

- Multimodal LLMs / vision-language models / [Document understanding and multimodal RAG](https://mlsanity.com/mllm/document-understanding-and-multimodal-rag/)

## Builds on (most cited first, 29 of 29)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [SMIL: Multimodal Learning with Severely Missing Modality](https://mlsanity.com/paper/2103.05677/) (2021)
- [Are Multimodal Transformers Robust to Missing Modality?](https://mlsanity.com/paper/2204.05454/) (2022)
- [VBPR: Visual Bayesian Personalized Ranking from Implicit Feedback](https://mlsanity.com/paper/1510.01784/) (2015)
- [Towards Universal Sequence Representation Learning for Recommender Systems](https://mlsanity.com/paper/2206.05941/) (2022)
- [Multimodal Prompting with Missing Modalities for Visual Recognition](https://mlsanity.com/paper/2303.03369/) (2023)
- [Bootstrap Latent Representations for Multi-modal Recommendation](https://mlsanity.com/paper/2207.05969/) (2022)
- [A Tale of Two Graphs: Freezing and Denoising Graph Structures for Multimodal Recommendation](https://mlsanity.com/paper/2211.06924/) (2022)
- [GeneCIS: A Benchmark for General Conditional Image Similarity](https://mlsanity.com/paper/2306.07969/) (2023)
- [TransModality: An End2End Fusion Method with Transformer for Multimodal Sentiment Analysis](https://mlsanity.com/paper/2009.02902/) (2020)
- [Multi-Domain Image Completion for Random Missing Input Data](https://mlsanity.com/paper/2007.05534/) (2020)
- [Exploring Adapter-based Transfer Learning for Recommender Systems: Empirical Studies and Practical Insights](https://mlsanity.com/paper/2305.15036/) (2023)
- [IISAN: Efficiently Adapting Multimodal Representation for Sequential Recommendation with Decoupled PEFT](https://mlsanity.com/paper/2404.02059/) (2024)
- [Knowledge Perceived Multi-modal Pretraining in E-commerce](https://mlsanity.com/paper/2109.00895/) (2021)
- [Efficient Unsupervised Sentence Compression by Fine-tuning Transformers with Reinforcement Learning](https://mlsanity.com/paper/2205.08221/) (2022)
- [MMRec: Simplifying Multimodal Recommendation](https://mlsanity.com/paper/2302.03497/) (2023)
- [Hierarchical Time-Aware Mixture of Experts for Multi-Modal Sequential Recommendation](https://mlsanity.com/paper/2501.14269/) (2025)
- [Knowledge Bridger: Towards Training-free Missing Modality Completion](https://mlsanity.com/paper/2502.19834/) (2025)
- [LRMM: Learning to Recommend with Missing Modalities](https://mlsanity.com/paper/1808.06791/) (2018)
- [Do We Really Need to Drop Items with Missing Modalities in Multimodal Recommendation?](https://mlsanity.com/paper/2408.11767/) (2024)
- [Efficient and Effective Adaptation of Multimodal Foundation Models in Sequential Recommendation](https://mlsanity.com/paper/2411.02992/) (2024)
- [Disentangling and Generating Modalities for Recommendation in Missing Modality Scenarios](https://mlsanity.com/paper/2504.16352/) (2025)
- [A Survey of Automatic Evaluation Methods on Text, Visual and Speech Generations](https://mlsanity.com/paper/2506.10019/) (2025)
- [MCE: Towards a General Framework for Handling Missing Modalities under Imbalanced Missing Rates](https://mlsanity.com/paper/2510.10534/) (2025)
- [A^2-GCN: An Attribute-aware Attentive GCN Model for Recommendation](https://mlsanity.com/paper/2003.09086/) (2020)
- [LLMPopcorn: Exploring LLMs as Assistants for Popular Micro-video Generation](https://mlsanity.com/paper/2502.12945/) (2025)

## Built on by (most cited first, 1 of 1)

- [Sequential Modality Dropout for Robust Multi-Modal Sequential Recommendation](https://mlsanity.com/paper/2608.10240/) (2026)

## Abstract (arXiv)

Missing-modality information on e-commerce platforms, such as absent product images or textual descriptions, often arises from annotation errors or incomplete metadata, impairing both product presentation and downstream applications such as recommendation systems. Motivated by the multimodal generative capabilities of recent Multimodal Large Language Models (MLLMs), this work investigates a fundamental yet underexplored question: can MLLMs generate missing modalities for products in e-commerce scenarios? We propose the Missing Modality Product Completion Benchmark (MMPCBench), which consists of two sub-benchmarks: a Content Quality Completion Benchmark and a Recommendation Benchmark. We further evaluate six state-of-the-art MLLMs from the Qwen2.5-VL and Gemma-3 model families across nine real-world e-commerce categories, focusing on image-to-text and text-to-image completion tasks. Experimental results show that while MLLMs can capture high-level semantics, they struggle with fine-grained word-level and pixel- or patch-level alignment. In addition, performance varies substantially across product categories and model scales, and we observe no trivial correlation between model size and performance, in contrast to trends commonly reported in mainstream benchmarks. We also explore Group Relative Policy Optimization (GRPO) to better align MLLMs with this task. GRPO improves image-to-text completion but does not yield gains for text-to-image completion. Overall, these findings expose the limitations of current MLLMs in real-world cross-modal generation and represent an early step toward more effective missing-modality product completion.