UniGame: Turning a Unified Multimodal Model Into Its Own Adversary
Not on the map. Why?
Abstract (arXiv)
Unified Multimodal Models (UMMs) have shown impressive performance in both understanding and generation with a single architecture. However, UMMs still exhibit a fundamental inconsistency: understanding favors compact embeddings, whereas generation favors reconstruction-rich representations. This structural trade-off produces misaligned decision boundaries, degraded cross-modal coherence, and heightened vulnerability under distributional and adversarial shifts. In this paper, we present UniGame, a self-adversarial post-training framework that directly targets the inconsistencies. By applying a lightweight perturber at the shared token interface, UniGame enables the generation branch to actively seek and challenge fragile understanding, turning the model itself into its own adversary. Experiments demonstrate that UniGame significantly improves the consistency (+4.6%). Moreover, it also achieves substantial improvements in understanding (+3.6%), generation (+0.02)on GenEval, out-of-distribution and adversarial robustness (+4.8% and +6.2% on NaturalBench and AdVQA). The framework is architecture-agnostic, introduces less than 1% additional parameters, and is complementary to existing post-training methods. These results position adversarial self-play as a general and effective principle for enhancing the coherence, stability, and unified competence of future multimodal foundation models. The official code is available at: https://github.com/AIFrontierLab/TorchUMM
Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 42 of its 46 references that are in the map. First posted on arXiv 24 Nov 2025.
Builds on
42 of 46 references.
References · 42
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
- 2025Qwen3 Technical Reportcited by 8,195
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
- 2014Explaining and Harnessing Adversarial Examplescited by 7,287
- 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
- 2022Generative Adversarial Networkscited by 5,603
- 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
- 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
- 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,841
- 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,498
30 more
- 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,105
- 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,048
- 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
- 2023Evaluating Object Hallucination in Large Vision-Language Modelscited by 1,696
- 2025gpt-oss-120b & gpt-oss-20b Model Cardcited by 1,258
- 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignmentcited by 946
- 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 904
- 2025Emerging Properties in Unified Multimodal Pretrainingcited by 855
- 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 787
- 2024Emu3: Next-Token Prediction is All You Needcited by 761
- 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 734
- 2024Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generationcited by 447
- 2024OmniGen: Unified Image Generationcited by 388
- 2025BLIP3-o: A Family of Fully Open Unified Multimodal Models-Architecture, Training and Datasetcited by 363
- 2024SEED-X: Multimodal Models with Unified Multi-granularity Comprehension and Generationcited by 328
- 2020Large-Scale Adversarial Training for Vision-and-Language Representation Learningcited by 327
- 2023DreamSim: Learning New Dimensions of Human Visual Similarity using Synthetic Datacited by 322
- 2024TokenFlow: Unified Image Tokenizer for Multimodal Understanding and Generationcited by 200
- 2025WISE: A World Knowledge-Informed Semantic Evaluation for Text-to-Image Generationcited by 196
- 2025T2I-R1: Reinforcing Image Generation with Collaborative Semantic-level and Token-level CoTcited by 162
- 2024Group Robust Preference Optimization in Reward-free RLHFcited by 94
- 2024NaturalBench: Evaluating Vision-Language Models on Natural Adversarial Samplescited by 94
- 2024Dual Diffusion for Unified Image Generation and Understandingcited by 76
- 2021Adversarial VQA: A New Benchmark for Evaluating the Robustness of VQA Modelscited by 74
- 2025Harmonizing Visual Representations for Unified Multimodal Understanding and Generationcited by 74
- 2024Liquid: Language Models are Scalable and Unified Multi-modal Generatorscited by 62
- 2025Reconstruction Alignment Improves Unified Multimodal Modelscited by 48
- 2025Understanding Multimodal LLMs Under Distribution Shifts: An Information-Theoretic Approachcited by 15
- 2025Token Perturbation Guidance for Diffusion Modelscited by 14
- 2025Visual Instruction Bottleneck Tuningcited by 5
Built on by
Citing papers per year: 2026: 9.
9 papers.
- 2026TorchUMM: A Unified Multimodal Model Codebase for Evaluation, Analysis, and Post-trainingcited by 2
- 2026LatentUMM: Dual Latent Alignment for Unified Multimodal Modelscited by 1
- 2026IMUG-Bench: Benchmarking Unified Multimodal Models on Interleaved Understanding and Generationcited by 0
- 2026Automatic Hard Example Synthesis with Multi-Level Agentic Data Curationcited by 0
- 2026Cross-Branch Conflict as a Shield: Safeguarding Facial Identities in Unified Multimodal Image Editingcited by 0
- 2026Do Unified Multimodal Models Think in One Space? A Lens Through Cross-Branch Steeringcited by 0
- 2026Mutually Adversarial Self-Training with Evolving Data for Unified Multimodal Modelscited by 0
- 2026Does Gradient Conflict Predict the Understanding--Generation Trade-off? A Controlled Audit of Conflict-Metric Validity in Unified Multimodal Modelscited by 0
- 2026Beyond Layers: Position-Resolved Gradient Conflict and Position-Aware Modulation for Unified Multimodal Modelscited by 0
- 2026Beyond Layers: Position-Resolved Gradient Conflict and Position-Aware Modulation for Unified Multimodal Models29 Sep 2026
- 2026Does Gradient Conflict Predict the Understanding--Generation Trade-off? A Controlled Audit of Conflict-Metric Validity in Unified Multimodal Models29 Sep 2026
- 2026Mutually Adversarial Self-Training with Evolving Data for Unified Multimodal Models28 Sep 2026
- 2026Do Unified Multimodal Models Think in One Space? A Lens Through Cross-Branch Steering29 Jul 2026
- 2026Cross-Branch Conflict as a Shield: Safeguarding Facial Identities in Unified Multimodal Image Editing18 Jul 2026
- 2026Automatic Hard Example Synthesis with Multi-Level Agentic Data Curation15 Jul 2026
- 2026IMUG-Bench: Benchmarking Unified Multimodal Models on Interleaved Understanding and Generation8 Jun 2026
- 2026LatentUMM: Dual Latent Alignment for Unified Multimodal Models18 May 2026
- 2026TorchUMM: A Unified Multimodal Model Codebase for Evaluation, Analysis, and Post-training12 Apr 2026