# ELMM: Efficient Lightweight Multimodal Large Language Models for Multimodal Knowledge Graph Completion

arXiv: https://arxiv.org/abs/2510.16753 · HTML: https://mlsanity.com/paper/2510.16753/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Document understanding and multimodal RAG, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 20 of its 47 references that are in the map.
- First posted on arXiv 19 Oct 2025.

## Placed in

- Multimodal LLMs / vision-language models / [Document understanding and multimodal RAG](https://mlsanity.com/mllm/document-understanding-and-multimodal-rag/)

## Builds on (most cited first, 20 of 20)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Improved Baselines with Visual Instruction Tuning](https://mlsanity.com/paper/2310.03744/) (2023)
- [VisualBERT: A Simple and Performant Baseline for Vision and Language](https://mlsanity.com/paper/1908.03557/) (2019)
- [VL-BERT: Pre-training of Generic Visual-Linguistic Representations](https://mlsanity.com/paper/1908.08530/) (2019)
- [Generative Multimodal Models are In-Context Learners](https://mlsanity.com/paper/2312.13286/) (2023)
- [KG-BERT: BERT for Knowledge Graph Completion](https://mlsanity.com/paper/1909.03193/) (2019)
- [A Survey on Knowledge Graph-Based Recommender Systems](https://mlsanity.com/paper/2003.00911/) (2020)
- [Knowledge Graph Prompting for Multi-Document Question Answering](https://mlsanity.com/paper/2308.11730/) (2023)
- [MobileVLM : A Fast, Strong and Open Vision Language Assistant for Mobile Devices](https://mlsanity.com/paper/2312.16886/) (2023)
- [MMKG: Multi-Modal Knowledge Graphs](https://mlsanity.com/paper/1903.05485/) (2019)
- [EfficientVLA: Training-Free Acceleration and Compression for Vision-Language-Action Models](https://mlsanity.com/paper/2506.10100/) (2025)
- [Image-embodied Knowledge Representation Learning](https://mlsanity.com/paper/1609.07028/) (2016)
- [DiffKG: Knowledge Graph Diffusion Model for Recommendation](https://mlsanity.com/paper/2312.16890/) (2023)
- [LGMRec: Local and Global Graph Learning for Multimodal Recommendation](https://mlsanity.com/paper/2312.16400/) (2023)
- [FineCLIPER: Multi-modal Fine-grained CLIP for Dynamic Facial Expression Recognition with AdaptERs](https://mlsanity.com/paper/2407.02157/) (2024)
- [IMF: Interactive Multimodal Fusion Model for Link Prediction](https://mlsanity.com/paper/2303.10816/) (2023)
- [NativE: Multi-modal Knowledge Graph Completion in the Wild](https://mlsanity.com/paper/2406.17605/) (2024)
- [Forecasting high-impact research topics via machine learning on evolving knowledge graphs](https://mlsanity.com/paper/2402.08640/) (2024)
- [MKGL: Mastery of a Three-Word Language](https://mlsanity.com/paper/2410.07526/) (2024)
- [UniAttn: Reducing Inference Costs via Softmax Unification for Post-Training LLMs](https://mlsanity.com/paper/2502.00439/) (2025)

## Built on by (most cited first, 1 of 1)

- [Mario: Multimodal Graph Reasoning with Large Language Models](https://mlsanity.com/paper/2603.05181/) (2026)

## Abstract (arXiv)

Multimodal Knowledge Graphs (MKGs) extend traditional knowledge graphs by incorporating visual and textual modalities, enabling richer and more expressive entity representations. However, existing MKGs often suffer from incompleteness, which hinder their effectiveness in downstream tasks. Therefore, multimodal knowledge graph completion (MKGC) task is receiving increasing attention. While large language models (LLMs) have shown promise for knowledge graph completion (KGC), their application to the multimodal setting remains underexplored. Moreover, applying Multimodal Large Language Models (MLLMs) to the task of MKGC introduces significant challenges: (1) the large number of image tokens per entity leads to semantic noise and modality conflicts, and (2) the high computational cost of processing large token inputs. To address these issues, we propose Efficient Lightweight Multimodal Large Language Models (ELMM) for MKGC. ELMM proposes a Multi-view Visual Token Compressor (MVTC) based on multi-head attention mechanism, which adaptively compresses image tokens from both textual and visual views, thereby effectively reducing redundancy while retaining necessary information and avoiding modality conflicts. Additionally, we design an attention pruning strategy to remove redundant attention layers from MLLMs, thereby significantly reducing the inference cost. We further introduce a linear projection to compensate for the performance degradation caused by pruning. Extensive experiments on four benchmark datasets demonstrate that ELMM achieves state-of-the-art performance.