Do Text Edits Generalize to Visual Generation? Benchmarking Cross-Modal Knowledge Editing in UMMs
Not on the map. Why?
Abstract (arXiv)
Unified multimodal models (UMMs) have emerged as a promising paradigm for general-purpose multimodal intelligence. As they are deployed in real-world applications, effectively updating internal knowledge becomes critical. While knowledge editing has matured for text-only models, it remains unclear whether edits that successfully modify textual outputs also transfer to image generation in UMMs. To study this question, we introduce UniKE, the first benchmark for cross-modality knowledge editing in UMMs, comprising 2,971 edit subjects spanning attribute and relation edits. Using VQA-based visual verification, we reveal a striking modality gap: text-side efficacy can reach approximately 92%, whereas the best overall VQA accuracy under direct image generation is only 18.5%. We further propose Reasoning-augmented Parameter Editing, which explicitly activates edited knowledge before generation and improves overall VQA accuracy for all evaluated model-editor pairs, with gains up to 18.6 percentage points. Mechanistic analysis shows that this gap is associated with partial alignment between edited textual representations and the conditioning pathways for visual generation, where edits sufficient for text outputs may remain too weak or misaligned to steer image synthesis. These findings show that textual knowledge edits do not guarantee reliable cross-modality transfer and motivate modality-aware editing methods. Our code and data are available at https://github.com/gxx27/UniKE.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 31 of its 47 references that are in the map. First posted on arXiv 30 May 2026.
Builds on
31 of 47 references.
References · 31
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
- 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
- 2022Locating and Editing Factual Associations in GPTcited by 2,609
- 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,173
- 2022Mass-Editing Memory in a Transformercited by 976
- 2025Emerging Properties in Unified Multimodal Pretrainingcited by 855
- 2024Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMscited by 719
- 2023Visual ChatGPT: Talking, Drawing and Editing with Visual Foundation Modelscited by 659
- 2024Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Modelcited by 468
- 2017Zero-Shot Relation Extraction via Reading Comprehensioncited by 425
- 2025BLIP3-o: A Family of Fully Open Unified Multimodal Models-Architecture, Training and Datasetcited by 363
19 more
- 2023MQuAKE: Assessing Knowledge Editing in Language Models via Multi-Hop Questionscited by 292
- 2023Can We Edit Factual Knowledge by In-Context Learning?cited by 284
- 2023LLM-grounded Diffusion: Enhancing Prompt Understanding of Text-to-Image Diffusion Models with Large Language Modelscited by 242
- 2024AlphaEdit: Null-Space Constrained Knowledge Editing for Language Modelscited by 224
- 2023Knowledge Editing for Large Language Models: A Surveycited by 189
- 2023PMET: Precise Model Editing in a Transformercited by 170
- 2024A Comprehensive Study of Knowledge Editing for Large Language Modelscited by 157
- 2023Editing Implicit Assumptions in Text-to-Image Diffusion Modelscited by 128
- 2023The Truth is in There: Improving Reasoning in Language Models with Layer-Selective Rank Reductioncited by 120
- 2024A Thorough Examination of Decoding Methods in the Era of LLMscited by 112
- 2025Unified Multimodal Understanding and Generation Models: Advances, Challenges, and Opportunitiescited by 81
- 2025Unlocking Multimodal Mathematical Reasoning via Process Reward Modelcited by 64
- 2023Localizing and Editing Knowledge in Text-to-Image Generative Modelscited by 52
- 2023ReFACT: Updating Text-to-Image Models by Editing the Text Encodercited by 32
- 2025ROVER: Benchmarking Reciprocal Cross-Modal Reasoning for Omnimodal Generationcited by 13
- 2026Are VLMs Seeing or Just Saying? Uncovering the Illusion of Visual Re-examinationcited by 12
- 2026From Narrow to Panoramic Vision: Attention-Guided Cold-Start Reshapes Multimodal Reasoningcited by 8
- 2024The Narrow Gate: Localized Image-Text Communication in Native Multimodal Modelscited by 4
- 2026Asymmetric Idiosyncrasies in Multimodal Modelscited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.