mlsanityData to 14 Oct 2026

Unified Personalized Understanding, Generating and Editing

arXiv 2601.06965 ↗First posted 11 Jan 2026cs.CVcited by 8

On the map: Multimodal LLMs / vision-language models → MLLM training and visual instruction tuning

Abstract (arXiv)

Unified large multimodal models (LMMs) have achieved remarkable progress in general-purpose multimodal understanding and generation. However, they still operate under a ``one-size-fits-all'' paradigm and struggle to model user-specific concepts (e.g., generate a photo of texttt<maeve>) in a consistent and controllable manner. Existing personalization methods typically rely on external retrieval, which is inefficient and poorly integrated into unified multimodal pipelines. Recent personalized unified models introduce learnable soft prompts to encode concept information, yet they either couple understanding and generation or depend on complex multi-stage training, leading to cross-task interference and ultimately to fuzzy or misaligned personalized knowledge. We present OmniPersona, an end-to-end personalization framework for unified LMMs that, for the first time, integrates personalized understanding, generation, and image editing within a single architecture. OmniPersona introduces structurally decoupled concept tokens, allocating dedicated subspaces for different tasks to minimize interference, and incorporates an explicit knowledge replay mechanism that propagates personalized attribute knowledge across tasks, enabling consistent personalized behavior. To systematically evaluate unified personalization, we propose textbftextttOmniPBench, extending the public UnifyBench concept set with personalized editing tasks and cross-task evaluation protocols integrating understanding, generation, and editing. Experimental results demonstrate that OmniPersona delivers competitive and robust performance across diverse personalization tasks. We hope OmniPersona will serve as a strong baseline and spur further research on controllable, unified personalization.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 39 of its 40 references that are in the map. First posted on arXiv 11 Jan 2026.

Builds on

39 of 40 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025PixelRefer: A Unified Framework for Spatio-Temporal Object Referring with Arbitrary Granularity27 Oct
  2. 2025YoChameleon: Personalized Vision and Language Generation29 Apr
  3. 2024RAP: Retrieval-Augmented Personalization for Multimodal Large Language Models17 Oct
  4. 2024Yo'LLaVA: Your Personalized Language and Vision Assistant13 Jun
  5. 2024HyperLLaVA: Dynamic Visual and Language Expert Tuning for Multimodal Large Language Models20 Mar

Other references · 34

  1. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,300
  2. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,245
  3. 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Modelscited by 1,464
  4. 2025Qwen-Image Technical Reportcited by 1,044
  5. 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 927
  6. 2025Emerging Properties in Unified Multimodal Pretrainingcited by 897
  7. 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 812
  8. 2024Emu3: Next-Token Prediction is All You Needcited by 781
  9. 2024Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generationcited by 457
  10. 2023LaMP: When Large Language Models Meet Personalizationcited by 437
  11. 2025BLIP3-o: A Family of Fully Open Unified Multimodal Models-Architecture, Training and Datasetcited by 376
  12. 2023InstantBooth: Personalized Text-to-Image Generation without Test-Time Finetuningcited by 345
22 more
  1. 2024SEED-X: Multimodal Models with Unified Multi-granularity Comprehension and Generationcited by 331
  2. 2023Unified-IO 2: Scaling Autoregressive Multimodal Models with Vision, Language, Audio, and Actioncited by 302
  3. 2024VILA-U: a Unified Foundation Model Integrating Visual Understanding and Generationcited by 295
  4. 2025Show-o2: Improved Native Unified Multimodal Modelscited by 250
  5. 2025Transfer between Modalities with MetaQueriescited by 236
  6. 2024MetaMorph: Multimodal Understanding and Generation via Instruction Tuningcited by 206
  7. 2025UniTok: A Unified Tokenizer for Visual Generation and Understandingcited by 153
  8. 2025HealthGPT: A Medical Large Vision-Language Model for Unifying Comprehension and Generation via Heterogeneous Knowledge Adaptationcited by 114
  9. 2025Unified Multimodal Understanding and Generation Models: Advances, Challenges, and Opportunitiescited by 85
  10. 2024VideoRefer Suite: Advancing Spatial-Temporal Object Understanding with Video LLMcited by 72
  11. 2024MyVLM: Personalizing VLMs for User-Specific Queriescited by 58
  12. 2025UniCTokens: Boosting Personalized Understanding and Generation via Unified Concept Tokenscited by 56
  13. 2022BoostMIS: Boosting Medical Image Semi-supervised Learning with Adaptive Pseudo Labeling and Informative Active Annotationcited by 54
  14. 2024MC-LLaVA: Multi-Concept Personalized Vision-Language Modelcited by 54
  15. 2025UniToken: Harmonizing Multimodal Understanding and Generation through Unified Visual Encodingcited by 49
  16. 2022DUET: A Tuning-Free Device-Cloud Collaborative Parameters Generation Framework for Efficient Device Model Generalizationcited by 35
  17. 2024Imagine yourself: Tuning-Free Personalized Image Generationcited by 30
  18. 2021Consensus Graph Representation Learning for Better Grounded Image Captioningcited by 28
  19. 2025EOC-Bench: Can MLLMs Identify, Recall, and Forecast Objects in an Egocentric World?cited by 21
  20. 2023Revisiting the Domain Shift and Sample Uncertainty in Multi-source Active Domain Transfercited by 20
  21. 2025WiseEdit: Benchmarking Cognition- and Creativity-Informed Image Editingcited by 11
  22. 2024MAKIMA: Tuning-free Multi-Attribute Open-domain Video Editing via Mask-Guided Attention Modulationcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 8.

8 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025MM-R1: Unleashing the Power of Unified Multimodal Large Language Models for Personalized Image Generation
  2. 2025OmniMamba: Efficient and Unified Multimodal Understanding and Generation via State Space Models
  3. 2026Omni-R1: Towards the Unified Generative Paradigm for Multimodal Reasoning
  4. 2025MANZANO: A Simple and Scalable Unified Multimodal Model with a Hybrid Vision Tokenizer
  5. 2025UniModel: A Visual-Only Framework for Unified Multimodal Understanding and Generation
  6. 2026Context Unrolling in Omni Models
  7. 2026UniRect-CoT: Enhancing Generation in Unified Multimodal Models via Reflective Rectification with Inherent Understanding
  8. 2026Omni-Persona: Systematic Benchmarking and Improving Omnimodal Personalization