# RAGAR: Retrieval Augmented Personalized Image Generation Guided by Recommendation

arXiv: https://arxiv.org/abs/2505.01657 · HTML: https://mlsanity.com/paper/2505.01657/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal RAG, one of 15 directions in Retrieval-augmented generation.
- Builds on 23 of its 39 references that are in the map.
- First posted on arXiv 3 May 2025.

## Placed in

- Retrieval-augmented generation / [Multimodal RAG](https://mlsanity.com/rag/multimodal-rag/)

## Builds on (most cited first, 23 of 23)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion](https://mlsanity.com/paper/2208.01618/) (2022)
- [Improving language models by retrieving from trillions of tokens](https://mlsanity.com/paper/2112.04426/) (2021)
- [Generating Images with Multimodal Language Models](https://mlsanity.com/paper/2305.17216/) (2023)
- [GPT-4V(ision) is a Human-Aligned Evaluator for Text-to-3D Generation](https://mlsanity.com/paper/2401.04092/) (2024)
- [WISA: World Simulator Assistant for Physics-Aware Text-to-Video Generation](https://mlsanity.com/paper/2503.08153/) (2025)
- [Unified Language-Vision Pretraining in LLM with Dynamic Discrete Visual Tokenization](https://mlsanity.com/paper/2309.04669/) (2023)
- [POG: Personalized Outfit Generation for Fashion Recommendation at Alibaba iFashion](https://mlsanity.com/paper/1905.01866/) (2019)
- [VIP5: Towards Multimodal Foundation Models for Recommendation](https://mlsanity.com/paper/2305.14302/) (2023)
- [Diffusion Models for Generative Outfit Recommendation](https://mlsanity.com/paper/2402.17279/) (2024)
- [DropoutGS: Dropping Out Gaussians for Better Sparse-view Rendering](https://mlsanity.com/paper/2504.09491/) (2025)
- [Latent Structure Mining with Contrastive Modality Fusion for Multimedia Recommendation](https://mlsanity.com/paper/2111.00678/) (2021)
- [RelaCtrl: Relevance-Guided Efficient Control for Diffusion Transformers](https://mlsanity.com/paper/2502.14377/) (2025)
- [Towards Highly Realistic Artistic Style Transfer via Stable Diffusion with Step-aware and Layer-aware Prompt](https://mlsanity.com/paper/2404.11474/) (2024)
- [FancyVideo: Towards Dynamic and Consistent Video Generation via Cross-frame Textual Guidance](https://mlsanity.com/paper/2408.08189/) (2024)
- [U-StyDiT: Ultra-high Quality Artistic Style Transfer Using Diffusion Transformers](https://mlsanity.com/paper/2503.08157/) (2025)
- [AdBooster: Personalized Ad Creative Generation using Stable Diffusion Outpainting](https://mlsanity.com/paper/2309.11507/) (2023)

## Built on by (most cited first, 8 of 8)

- [Lay2Story: Extending Diffusion Transformers for Layout-Togglable Story Generation](https://mlsanity.com/paper/2508.08949/) (2025)
- [InnoAds-Composer: Efficient Condition Composition for E-Commerce Poster Generation](https://mlsanity.com/paper/2603.05898/) (2026)
- [Premier: Personalized Preference Modulation with Learnable User Embedding in Text-to-Image Generation](https://mlsanity.com/paper/2603.20725/) (2026)
- [Design Your Ad: Personalized Advertising Image and Text Generation with Unified Autoregressive Models](https://mlsanity.com/paper/2605.12138/) (2026)
- [FedCVU: Federated Learning for Cross-View Video Understanding](https://mlsanity.com/paper/2603.21647/) (2026)
- [TailorMind: Towards Preference-Aligned Multimodal Content Generation](https://mlsanity.com/paper/2606.23643/) (2026)
- [Navigating User Behavior toward Personalized Multimodal Generation](https://mlsanity.com/paper/2606.24196/) (2026)
- [Memory-Guided B-Roll Generation from User Video Collections](https://mlsanity.com/paper/2610.01884/) (2026)

## Abstract (arXiv)

Personalized image generation is crucial for improving the user experience, as it renders reference images into preferred ones according to user visual preferences. Although effective, existing methods face two main issues. First, existing methods treat all items in the user historical sequence equally when extracting user preferences, overlooking the varying semantic similarities between historical items and the reference item. Disproportionately high weights for low-similarity items distort users' visual preferences for the reference item. Second, existing methods heavily rely on consistency between generated and reference images to optimize the generation, which leads to underfitting user preferences and hinders personalization. To address these issues, we propose Retrieval Augment Personalized Image GenerAtion guided by Recommendation (RAGAR). Our approach uses a retrieval mechanism to assign different weights to historical items according to their similarities to the reference item, thereby extracting more refined users' visual preferences for the reference item. Then we introduce a novel rank task based on the multi-modal ranking model to optimize the personalization of the generated images instead of forcing depend on consistency. Extensive experiments and human evaluations on three real-world datasets demonstrate that RAGAR achieves significant improvements in both personalization and semantic metrics compared to five baselines.