# MORE: Multi-mOdal REtrieval Augmented Generative Commonsense Reasoning

arXiv: https://arxiv.org/abs/2402.13625 · HTML: https://mlsanity.com/paper/2402.13625/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal reasoning, one of 17 directions in LLM reasoning.
- Builds on 8 of its 53 references that are in the map.
- First posted on arXiv 21 Feb 2024.

## Placed in

- LLM reasoning / [Multimodal reasoning](https://mlsanity.com/lr/multimodal-reasoning/)
- Retrieval-augmented generation / [Multimodal RAG](https://mlsanity.com/rag/multimodal-rag/)

## Builds on (most cited first, 8 of 8)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [Atlas: Few-shot Learning with Retrieval Augmented Language Models](https://mlsanity.com/paper/2208.03299/) (2022)
- [REPLUG: Retrieval-Augmented Black-Box Language Models](https://mlsanity.com/paper/2301.12652/) (2023)
- [RECOMP: Improving Retrieval-Augmented LMs with Compression and Selective Augmentation](https://mlsanity.com/paper/2310.04408/) (2023)
- [TCRA-LLM: Token Compression Retrieval Augmented Large Language Model for Inference Cost Reduction](https://mlsanity.com/paper/2310.15556/) (2023)
- [Visualize Before You Write: Imagination-Guided Open-Ended Text Generation](https://mlsanity.com/paper/2210.03765/) (2022)
- [Optimizing Retrieval-augmented Reader Models via Token Elimination](https://mlsanity.com/paper/2310.13682/) (2023)

## Built on by (most cited first, 12 of 12)

- [Retrieval-Augmented Generation with Graphs (GraphRAG)](https://mlsanity.com/paper/2501.00309/) (2024)
- [MRAG-Bench: Vision-Centric Evaluation for Retrieval-Augmented Multimodal Models](https://mlsanity.com/paper/2410.08182/) (2024)
- [Ask in Any Modality: A Comprehensive Survey on Multimodal Retrieval-Augmented Generation](https://mlsanity.com/paper/2502.08826/) (2025)
- [UniversalRAG: Retrieval-Augmented Generation over Corpora of Diverse Modalities and Granularities](https://mlsanity.com/paper/2504.20734/) (2025)
- [Retrieval Meets Reasoning: Even High-school Textbook Knowledge Benefits Multimodal Reasoning](https://mlsanity.com/paper/2405.20834/) (2024)
- [Benchmarking Retrieval-Augmented Generation in Multi-Modal Contexts](https://mlsanity.com/paper/2502.17297/) (2025)
- [Revisiting Compositional Generalization Capability of Large Language Models Considering Instruction Following Ability](https://mlsanity.com/paper/2506.15629/) (2025)
- [LaMI: Augmenting Large Language Models via Late Multi-Image Fusion](https://mlsanity.com/paper/2406.13621/) (2024)
- [Augmented Vision-Language Models: A Systematic Review](https://mlsanity.com/paper/2507.22933/) (2025)
- [From Scenes to Elements: Multi-Granularity Evidence Retrieval for Verifiable Multimodal RAG](https://mlsanity.com/paper/2605.15019/) (2026)
- [MES-RAG: Bringing Multi-modal, Entity-Storage, and Secure Enhancements to RAG](https://mlsanity.com/paper/2503.13563/) (2025)
- [Evolution of Multimodal Question Answering: From Modality-Adaptive Extraction to Unified Language Representation](https://mlsanity.com/paper/2609.08896/) (2026)

## Abstract (arXiv)

Since commonsense information has been recorded significantly less frequently than its existence, language models pre-trained by text generation have difficulty to learn sufficient commonsense knowledge. Several studies have leveraged text retrieval to augment the models' commonsense ability. Unlike text, images capture commonsense information inherently but little effort has been paid to effectively utilize them. In this work, we propose a novel Multi-mOdal REtrieval (MORE) augmentation framework, to leverage both text and images to enhance the commonsense ability of language models. Extensive experiments on the Common-Gen task have demonstrated the efficacy of MORE based on the pre-trained models of both single and multiple modalities.