mlsanityData to 7 Oct 2026

UniGame: Turning a Unified Multimodal Model Into Its Own Adversary

arXiv 2511.19413 ↗First posted 24 Nov 2025cs.LG cs.AI cs.CVcited by 9

Not on the map. Why?

Abstract (arXiv)

Unified Multimodal Models (UMMs) have shown impressive performance in both understanding and generation with a single architecture. However, UMMs still exhibit a fundamental inconsistency: understanding favors compact embeddings, whereas generation favors reconstruction-rich representations. This structural trade-off produces misaligned decision boundaries, degraded cross-modal coherence, and heightened vulnerability under distributional and adversarial shifts. In this paper, we present UniGame, a self-adversarial post-training framework that directly targets the inconsistencies. By applying a lightweight perturber at the shared token interface, UniGame enables the generation branch to actively seek and challenge fragile understanding, turning the model itself into its own adversary. Experiments demonstrate that UniGame significantly improves the consistency (+4.6%). Moreover, it also achieves substantial improvements in understanding (+3.6%), generation (+0.02)on GenEval, out-of-distribution and adversarial robustness (+4.8% and +6.2% on NaturalBench and AdVQA). The framework is architecture-agnostic, introduces less than 1% additional parameters, and is complementary to existing post-training methods. These results position adversarial self-play as a general and effective principle for enhancing the coherence, stability, and unified competence of future multimodal foundation models. The official code is available at: https://github.com/AIFrontierLab/TorchUMM

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 42 of its 46 references that are in the map. First posted on arXiv 24 Nov 2025.

Builds on

42 of 46 references.

References · 42

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2025Qwen3 Technical Reportcited by 8,195
  4. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  5. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  6. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  7. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  8. 2022Generative Adversarial Networkscited by 5,603
  9. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  10. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
  11. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,841
  12. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,498
30 more
  1. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,105
  2. 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,048
  3. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
  4. 2023Evaluating Object Hallucination in Large Vision-Language Modelscited by 1,696
  5. 2025gpt-oss-120b & gpt-oss-20b Model Cardcited by 1,258
  6. 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignmentcited by 946
  7. 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 904
  8. 2025Emerging Properties in Unified Multimodal Pretrainingcited by 855
  9. 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 787
  10. 2024Emu3: Next-Token Prediction is All You Needcited by 761
  11. 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 734
  12. 2024Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generationcited by 447
  13. 2024OmniGen: Unified Image Generationcited by 388
  14. 2025BLIP3-o: A Family of Fully Open Unified Multimodal Models-Architecture, Training and Datasetcited by 363
  15. 2024SEED-X: Multimodal Models with Unified Multi-granularity Comprehension and Generationcited by 328
  16. 2020Large-Scale Adversarial Training for Vision-and-Language Representation Learningcited by 327
  17. 2023DreamSim: Learning New Dimensions of Human Visual Similarity using Synthetic Datacited by 322
  18. 2024TokenFlow: Unified Image Tokenizer for Multimodal Understanding and Generationcited by 200
  19. 2025WISE: A World Knowledge-Informed Semantic Evaluation for Text-to-Image Generationcited by 196
  20. 2025T2I-R1: Reinforcing Image Generation with Collaborative Semantic-level and Token-level CoTcited by 162
  21. 2024Group Robust Preference Optimization in Reward-free RLHFcited by 94
  22. 2024NaturalBench: Evaluating Vision-Language Models on Natural Adversarial Samplescited by 94
  23. 2024Dual Diffusion for Unified Image Generation and Understandingcited by 76
  24. 2021Adversarial VQA: A New Benchmark for Evaluating the Robustness of VQA Modelscited by 74
  25. 2025Harmonizing Visual Representations for Unified Multimodal Understanding and Generationcited by 74
  26. 2024Liquid: Language Models are Scalable and Unified Multi-modal Generatorscited by 62
  27. 2025Reconstruction Alignment Improves Unified Multimodal Modelscited by 48
  28. 2025Understanding Multimodal LLMs Under Distribution Shifts: An Information-Theoretic Approachcited by 15
  29. 2025Token Perturbation Guidance for Diffusion Modelscited by 14
  30. 2025Visual Instruction Bottleneck Tuningcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 9.

9 papers.