mlsanityData to 7 Oct 2026

MoMA: Multimodal LLM Adapter for Fast Personalized Image Generation

arXiv 2404.05674 ↗First posted 8 Apr 2024cs.CVcited by 35

On the map: Multimodal LLMs / vision-language models → MLLM training and visual instruction tuning

also in 1 more
Abstract (arXiv)

In this paper, we present MoMA: an open-vocabulary, training-free personalized image model that boasts flexible zero-shot capabilities. As foundational text-to-image models rapidly evolve, the demand for robust image-to-image translation grows. Addressing this need, MoMA specializes in subject-driven personalized image generation. Utilizing an open-source, Multimodal Large Language Model (MLLM), we train MoMA to serve a dual role as both a feature extractor and a generator. This approach effectively synergizes reference image and text prompt information to produce valuable image features, facilitating an image diffusion model. To better leverage the generated features, we further introduce a novel self-attention shortcut method that efficiently transfers image features to an image diffusion model, improving the resemblance of the target object in generated images. Remarkably, as a tuning-free plug-and-play module, our model requires only a single reference image and outperforms existing methods in generating images with high detail fidelity, enhanced identity-preservation and prompt faithfulness. Our work is open-source, thereby providing universal access to these advancements.

Key facts. Cited by 35 papers in the map (data through 7 Oct 2026). Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 30 of its 38 references that are in the map. First posted on arXiv 8 Apr 2024.

Builds on

30 of 38 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Improved Baselines with Visual Instruction Tuning5 Oct

Other references · 29

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2023GPT-4 Technical Reportcited by 18,132
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  5. 2023Segment Anythingcited by 8,158
  6. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  7. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  8. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  9. 2022Scalable Diffusion Models with Transformerscited by 5,931
  10. 2022Classifier-Free Diffusion Guidancecited by 5,717
  11. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
  12. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
17 more
  1. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
  2. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
  3. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
  4. 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Modelscited by 1,439
  5. 2022Multi-Concept Customization of Text-to-Image Diffusioncited by 1,099
  6. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoiserscited by 921
  7. 2023CogVLM: Visual Expert for Pretrained Language Modelscited by 704
  8. 2023BLIP-Diffusion: Pre-trained Subject Representation for Controllable Text-to-Image Generation and Editingcited by 438
  9. 2024InstantID: Zero-shot Identity-Preserving Generation in Secondscited by 431
  10. 2023ELITE: Encoding Visual Concepts into Textual Embeddings for Customized Text-to-Image Generationcited by 414
  11. 2023AnyDoor: Zero-shot Object-level Image Customizationcited by 411
  12. 2023InstantBooth: Personalized Text-to-Image Generation without Test-Time Finetuningcited by 343
  13. 2023SVDiff: Compact Parameter Space for Diffusion Fine-Tuningcited by 338
  14. 2023Break-A-Scene: Extracting Multiple Concepts from a Single Imagecited by 230
  15. 2023Taming Encoder for Zero Fine-tuning Image Customization with Text-to-Image Diffusion Modelscited by 116
  16. 2023Kandinsky: an Improved Text-to-Image Synthesis with Image Prior and Latent Diffusioncited by 114
  17. 2023PhotoVerse: Tuning-Free Image Customization with Text-to-Image Diffusion Modelscited by 70

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 12; 2025: 19; 2026: 4.

35 papers.

Sort:

By area:

  1. 2024CamI2V: Camera-Controlled Image-to-Video Diffusion Modelcited by 115
  2. 2024Playground v3: Improving Text-to-Image Alignment with Deep-Fusion Large Language Modelscited by 113
  3. 2024MV-Adapter: Multi-view Consistent Image Generation Made Easycited by 111
  4. 2024Text4Seg: Reimagining Image Segmentation as Text Generationcited by 59
  5. 2025RealCam-I2V: Real-World Image-to-Video Generation with Interactive Complex Camera Controlcited by 54
  6. 2025CreatiDesign: A Unified Multi-Conditional Diffusion Transformer for Creative Graphic Designcited by 36
  7. 2024A Survey on Personalized Content Synthesis with Diffusion Modelscited by 29
  8. 2025Generating Multi-Image Synthetic Data for Text-to-Image Customizationcited by 25
  9. 2025Personalize Anything for Free with Diffusion Transformercited by 22
  10. 2025Personalized Image Generation with Deep Generative Models: A Decade Surveycited by 20
  11. 2024Personalized Multimodal Large Language Models: A Surveycited by 19
  12. 2025InstantCharacter: Personalize Any Characters with a Scalable Diffusion Transformer Frameworkcited by 19
23 more
  1. 2024RealCustom++: Representing Images as Real Textual Word for Real-Time Customizationcited by 16
  2. 2025PolyVivid: Vivid Multi-Subject Video Generation with Cross-Modal Interaction and Enhancementcited by 16
  3. 2024MagicTailor: Component-Controllable Personalization in Text-to-Image Diffusion Modelscited by 14
  4. 2024CustomContrast: A Multilevel Contrastive Perspective For Subject-Driven Text-to-Image Customizationcited by 13
  5. 2024Text-to-Image Synthesis: A Decade Surveycited by 9
  6. 2024UNIC-Adapter: Unified Image-instruction Adapter with Multi-modal Transformer for Image Generationcited by 9
  7. 2025Text4Seg++: Advancing Image Segmentation via Generative Language Modelingcited by 8
  8. 2025LiON-LoRA: Rethinking LoRA Fusion to Unify Controllable Spatial and Temporal Generation for Video Diffusioncited by 6
  9. 2025Have Multimodal Large Language Models (MLLMs) Really Learned to Tell the Time on Analog Clocks?cited by 4
  10. 2025Uncertainty-o: One Model-agnostic Framework for Unveiling Uncertainty in Large Multimodal Modelscited by 4
  11. 2025FlashEdit: Decoupling Speed, Structure, and Semantics for Precise Image Editingcited by 4
  12. 2025FlexIP: Dynamic Control of Preservation and Personality for Customized Image Generationcited by 3
  13. 2025Create Anything Anywhere: Layout-Controllable Personalized Diffusion Model for Multiple Subjectscited by 3
  14. 2025FreeCus: Free Lunch Subject-driven Customization in Diffusion Transformerscited by 3
  15. 2025SketchAgent: Generating Structured Diagrams from Hand-Drawn Sketchescited by 3
  16. 2025IMG: Calibrating Diffusion Models via Implicit Multimodal Guidancecited by 3
  17. 2026Tuning-free Visual Effect Transfer across Videoscited by 3
  18. 2026CollectionLoRA: Collecting 50 Effects in 1 LoRA via Multi-Teacher On-Policy Distillationcited by 3
  19. 2024Joint Embeddings for Graph Instruction Tuningcited by 2
  20. 2025Identity-Preserving Text-to-Image Generation via Dual-Level Feature Decoupling and Expert-Guided Fusioncited by 2
  21. 2026TorchUMM: A Unified Multimodal Model Codebase for Evaluation, Analysis, and Post-trainingcited by 2
  22. 2026ICG: Improving Cover Image Generation via MLLM-based Prompting and Personalized Preference Alignmentcited by 2
  23. 2025SpotDiff: Spotting and Disentangling Interference in Feature Space for Subject-Preserving Image Generationcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2025MM-R1: Unleashing the Power of Unified Multimodal Large Language Models for Personalized Image Generation
  2. 2024MUMU: Bootstrapping Multimodal Image Generation from Text-to-Image Data
  3. 2025YoChameleon: Personalized Vision and Language Generation
  4. 2023LLMGA: Multimodal Large Language Model based Generation Assistant
  5. 2025MANZANO: A Simple and Scalable Unified Multimodal Model with a Hybrid Vision Tokenizer
  6. 2025Dynamic Image Prompt Adapter for Scalable Zero-shot Personalized Text-to-Image Generation
  7. 2023Kosmos-G: Generating Images in Context with Multimodal Large Language Models
  8. 2025Personalized Text-to-Image Generation with Auto-Regressive Models