MMCORE: MultiModal COnnection with Representation Aligned Latent Embeddings
On the map: Diffusion models → Image generation, text-to-image and editing
Abstract (arXiv)
We present MMCORE, a unified framework designed for multimodal image generation and editing. MMCORE leverages a pre-trained Vision-Language Model (VLM) to predict semantic visual embeddings via learnable query tokens, which subsequently serve as conditioning signals for a diffusion model. This streamlined design effectively transfers the rich understanding and reasoning capabilities of VLMs into the visual generation process. By obviating the need for deep fusion between autoregressive and diffusion models or training from scratch, MMCORE significantly reduces computational overhead while maintaining high-fidelity synthesis. MMCORE seamlessly integrates text-to-image synthesis with interleaved image generation, demonstrating robust multimodal comprehension in complex scenarios such as spatial reasoning and visual grounding. Comprehensive evaluations indicate that MMCORE consistently outperforms state-of-the-art baselines across a broad spectrum of text-to-image and single/multi-image editing benchmarks.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 43 of its 46 references that are in the map. First posted on arXiv 21 Apr 2026.
Builds on
43 of 46 references.
Closest ancestors · 6
Same direction, newest first
- 2025BLIP3-o: A Family of Fully Open Unified Multimodal Models-Architecture, Training and Dataset14 May
- 2025Seedream 2.0: A Native Chinese-English Bilingual Image Generation Foundation Model10 Mar
- 2024SeedEdit: Align Image Re-Generation to Image Editing11 Nov
- 2024Emu3: Next-Token Prediction is All You Need27 Sep
- 2024Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction3 Apr
- 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding23 May
Other references · 37
- 2017Attention Is All You Needcited by 59,932
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
- 2020Denoising Diffusion Probabilistic Modelscited by 20,851
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2020Scaling Laws for Neural Language Modelscited by 6,411
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
- 2022Scalable Diffusion Models with Transformerscited by 5,931
- 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
- 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
- 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
25 more
- 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
- 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
- 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
- 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
- 2017Accurate, Large Minibatch SGD: Training ImageNet in 1 Hourcited by 1,479
- 2025SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Featurescited by 1,127
- 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 904
- 2025Emerging Properties in Unified Multimodal Pretrainingcited by 855
- 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 787
- 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 734
- 2021Multimodal Few-Shot Learning with Frozen Language Modelscited by 657
- 2024Autoregressive Image Generation without Vector Quantizationcited by 644
- 2024Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Modelcited by 468
- 2024Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generationcited by 447
- 2024Infinity: Scaling Bitwise AutoRegressive Modeling for High-Resolution Image Synthesiscited by 297
- 2025Diffusion Transformers with Representation Autoencoderscited by 275
- 2023PaLI-X: On Scaling up a Multilingual Vision and Language Modelcited by 254
- 2025Seedream 4.0: Toward Next-generation Multimodal Image Generationcited by 233
- 2025Transfer between Modalities with MetaQueriescited by 231
- 2024LMFusion: Adapting Pretrained Language Models for Multimodal Generationcited by 120
- 2025UniWorld-V1: High-Resolution Semantic Encoders for Unified Visual Understanding and Generationcited by 64
- 2025SeedEdit 3.0: Fast and High-Quality Generative Image Editingcited by 52
- 2025Skywork R1V2: Multimodal Hybrid Reinforcement Learning for Reasoningcited by 48
- 2025Is Nano Banana Pro a Low-Level Vision All-Rounder? A Comprehensive Evaluation on 14 Tasks and 40 Datasetscited by 19
- 2025LightFusion: A Light-weighted, Double Fusion Framework for Unified Multimodal Understanding and Generationcited by 9
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026Bernini: Latent Semantic Planning for Video Diffusion
- 2025Growing Visual Generative Capacity for Pre-Trained MLLMs
- 2025MammothModa2: A Unified AR-Diffusion Framework for Multimodal Understanding and Generation
- 2025Generative Multimodal Pretraining with Discrete Diffusion Timestep Tokens
- 2025Bifrost-1: Bridging Multimodal LLMs and Diffusion Models with Patch-level CLIP Latents
- 2025Query-Kontext: An Unified Multimodal Model for Image Generation and Editing
- 2025MANZANO: A Simple and Scalable Unified Multimodal Model with a Hybrid Vision Tokenizer
- 2024MMIE: Massive Multimodal Interleaved Comprehension Benchmark for Large Vision-Language Models