SenseNova-U1: Unifying Multimodal Understanding and Generation with NEO-unify Architecture
On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning
Abstract (arXiv)
Recent large vision-language models (VLMs) remain fundamentally constrained by a persistent dichotomy: understanding and generation are treated as distinct problems, leading to fragmented architectures, cascaded pipelines, and misaligned representation spaces. We argue that this divide is not merely an engineering artifact, but a structural limitation that hinders the emergence of native multimodal intelligence. Hence, we introduce SenseNova-U1, a native unified multimodal paradigm built upon NEO-unify, in which understanding and generation evolve as synergistic views of a single underlying process. We launch two native unified variants, SenseNova-U1-8B-MoT and SenseNova-U1-A3B-MoT, built on dense (8B) and mixture-of-experts (30B-A3B) understanding baselines, respectively. Designed from first principles, they rival top-tier understanding-only VLMs across text understanding, vision-language perception, knowledge reasoning, agentic decision-making, and spatial intelligence. Meanwhile, they deliver strong semantic consistency and visual fidelity, excelling in conventional or knowledge-intensive any-to-image (X2I) synthesis, complex text-rich infographic generation, and interleaved vision-language generation, with or without think patterns. Beyond performance, we show detailed model design, data preprocessing, pre-/post-training, and inference strategies to support community research. Last but not least, preliminary evidence demonstrates that our models extend beyond perception and generation, performing strongly in vision-language-action (VLA) and world model (WM) scenarios. This points toward a broader roadmap where models do not translate between modalities, but think and act across them in a native manner. Multimodal AI is no longer about connecting separate systems, but about building a unified one and trusting the necessary capabilities to emerge from within.
Key facts. Cited by 27 papers in the map (data through 7 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 132 of its 156 references that are in the map. First posted on arXiv 12 May 2026.
Builds on
132 of 156 references.
Closest ancestors · 9
Same direction, newest first
- 2025From Pixels to Words -- Towards Native Vision-Language Primitives at Scale16 Oct
- 2025EVEv2: Improved Baselines for Encoder-Free Vision-Language Models10 Feb
- 2024HoVLE: Unleashing the Power of Monolithic Vision-Language Models with Holistic Vision-Language Embedding20 Dec
- 2024SOLO: A Single Transformer for Scalable Vision-Language Modeling8 Jul
- 2024Unveiling Encoder-Free Vision-Language Models17 Jun
- 2024Are We on the Right Way for Evaluating Large Vision-Language Models?29 Mar
- 2024Measuring Multimodal Mathematical Reasoning with MATH-Vision Dataset22 Feb
- 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts3 Oct
- 2022Flamingo: a Visual Language Model for Few-Shot Learning29 Apr
Other references · 123
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
- 2014Microsoft COCO: Common Objects in Contextcited by 17,498
- 2025Qwen3 Technical Reportcited by 8,195
- 2023Visual Instruction Tuningcited by 7,792
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
- 2017Neural Discrete Representation Learningcited by 4,650
- 2024GPT-4o System Cardcited by 4,467
- 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
- 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
- 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
- 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,536
38 more
- 2025Qwen3-VL Technical Reportcited by 2,534
- 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,105
- 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
- 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,758
- 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
- 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,280
- 2023Instruction-Following Evaluation for Large Language Modelscited by 1,143
- 2025Qwen-Image Technical Reportcited by 973
- 2025FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Spacecited by 960
- 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignmentcited by 946
- 2025OpenAI GPT-5 System Cardcited by 910
- 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 904
- 2025Emerging Properties in Unified Multimodal Pretrainingcited by 855
- 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 787
- 2023EVA-CLIP: Improved Training Techniques for CLIP at Scalecited by 786
- 2024Emu3: Next-Token Prediction is All You Needcited by 761
- 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 734
- 2023C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Modelscited by 694
- 2024Improved Distribution Matching Distillation for Fast Image Synthesiscited by 648
- 2023NExT-GPT: Any-to-Any Multimodal LLMcited by 610
- 2025Flow-GRPO: Training Flow Matching Models via Online RLcited by 605
- 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spacescited by 580
- 2023HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Modelscited by 572
- 2025Kimi K2: Open Agentic Intelligencecited by 523
- 2024MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmarkcited by 494
- 2024Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Modelcited by 468
- 2023OCRBench: On the Hidden Mystery of OCR in Large Multimodal Modelscited by 466
- 2024ELLA: Equip Diffusion Models with LLM for Enhanced Semantic Alignmentcited by 455
- 2024Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generationcited by 447
- 2023Generative Multimodal Models are In-Context Learnerscited by 446
- 2025GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learningcited by 439
- 2021InfographicVQAcited by 430
- 2025Step1X-Edit: A Practical Framework for General Image Editingcited by 417
- 2024OmniGen: Unified Image Generationcited by 388
- 2025Reconstruction vs. Generation: Taming Optimization Dilemma in Latent Diffusion Modelscited by 376
- 2025BLIP3-o: A Family of Fully Open Unified Multimodal Models-Architecture, Training and Datasetcited by 363
- 2024SEED-X: Multimodal Models with Unified Multi-granularity Comprehension and Generationcited by 328
- 2024Infinity: Scaling Bitwise AutoRegressive Modeling for High-Resolution Image Synthesiscited by 297
50 most cited of 123 All 123 (JSON)
Built on by
Citing papers per year: 2026: 27.
27 papers.
Sort:
By area:
- 2026Toward Native Multimodal Modeling: A Roadmapcited by 6
- 2026Vision as Unified Multimodal Generationcited by 5
- 2026Representation Forcing for Bottleneck-Free Unified Multimodal Modelscited by 4
- 2026VBVR-Pro: A Scalable and Verifiable Suite for Native Visual Reasoningcited by 4
- 2026An Empirical Study of Training Pixel-Space Text-to-Image Diffusion Modelscited by 3
- 2026How and What to Imagine? Visual Thinking in Unified Multimodal Models for Cross-View Spatial Reasoningcited by 2
- 2026Show the Signal, Hide the Noise: Spectral Forcing for Pixel-Space Diffusioncited by 2
- 2026ERA: Entropy-Guided Visual Token Pruning with Rectified Attention for Efficient MLLMscited by 2
- 2026Uncovering Understanding-Generation Synergy in Native Unified Multimodal Models: From Representation, Task to Systemcited by 2
- 2026GEAR: Guided End-to-End AutoRegression for Image Synthesiscited by 1
- 2026WeGenBench: A Multidimensional Diagnostic Benchmark towards Text-to-Image Model Optimizationcited by 0
- 2026DisciplineGen-1M: A Large-Scale Dataset for Multidisciplinary Visual Generation and Editingcited by 0
15 more
- 2026Cross-Branch Conflict as a Shield: Safeguarding Facial Identities in Unified Multimodal Image Editingcited by 0
- 2026STBridge: Shared-Target Alignment for Bridging Understanding and Generation in UMMscited by 0
- 2026Test-Time Curriculum for Open-Set AIGC Detectioncited by 0
- 2026UniWorld-Design: From Pixel Generation to Layer-Native Designcited by 0
- 2026UniSpace: Unified Visual Representation and Scalable Multimodal Modelingcited by 0
- 2026WithEveryone: Unified Planning and Identity Grounding for Group Image Generationcited by 0
- 2026Editable Visual Designcited by 0
- 2026Beyond Generation and Accuracy: Diagnosing and Enhancing Visual Chain-of-Thought for Geometry Problem Solvingcited by 0
- 2026FLAT: Resampling Image and Text into 1D Flexible-Length Aligned Transmodal Tokens for Retrieval and Generationcited by 0
- 2026GameHorizon Suite: Multi-Horizon Data and Evaluation in Gameplaycited by 0
- 2026How Far Are We from Removing the Visual Encoder? Scaling Laws for Encoder-Free Multimodal Pretrainingcited by 0
- 2026PixelUMM: Encoder-Free Unified Image and Video Understanding and Generationcited by 0
- 2026Self-correction Optimization for Interleaved Multimodal Generationcited by 0
- 2026From Prompting to Composing: A Spatial Canvas Interface for Poster Generationcited by 0
- 2026Reasoning-Informed Visual Editingcited by 0
- 2026Reasoning-Informed Visual Editing8 Oct 2026
- 2026From Prompting to Composing: A Spatial Canvas Interface for Poster Generation8 Oct 2026
- 2026Self-correction Optimization for Interleaved Multimodal Generation7 Oct 2026
- 2026PixelUMM: Encoder-Free Unified Image and Video Understanding and Generation29 Sep 2026
- 2026How Far Are We from Removing the Visual Encoder? Scaling Laws for Encoder-Free Multimodal Pretraining28 Sep 2026
- 2026GameHorizon Suite: Multi-Horizon Data and Evaluation in Gameplay21 Sep 2026
- 2026FLAT: Resampling Image and Text into 1D Flexible-Length Aligned Transmodal Tokens for Retrieval and Generation15 Sep 2026
- 2026Beyond Generation and Accuracy: Diagnosing and Enhancing Visual Chain-of-Thought for Geometry Problem Solving11 Sep 2026
- 2026Editable Visual Design3 Sep 2026
- 2026Uncovering Understanding-Generation Synergy in Native Unified Multimodal Models: From Representation, Task to System1 Sep 2026
- 2026VBVR-Pro: A Scalable and Verifiable Suite for Native Visual Reasoning26 Aug 2026
- 2026WithEveryone: Unified Planning and Identity Grounding for Group Image Generation20 Aug 2026
15 more
- 2026An Empirical Study of Training Pixel-Space Text-to-Image Diffusion Models17 Aug 2026
- 2026UniSpace: Unified Visual Representation and Scalable Multimodal Modeling9 Aug 2026
- 2026UniWorld-Design: From Pixel Generation to Layer-Native Design4 Aug 2026
- 2026Test-Time Curriculum for Open-Set AIGC Detection1 Aug 2026
- 2026STBridge: Shared-Target Alignment for Bridging Understanding and Generation in UMMs19 Jul 2026
- 2026Cross-Branch Conflict as a Shield: Safeguarding Facial Identities in Unified Multimodal Image Editing18 Jul 2026
- 2026Vision as Unified Multimodal Generation7 Jul 2026
- 2026DisciplineGen-1M: A Large-Scale Dataset for Multidisciplinary Visual Generation and Editing2 Jul 2026
- 2026GEAR: Guided End-to-End AutoRegression for Image Synthesis30 Jun 2026
- 2026ERA: Entropy-Guided Visual Token Pruning with Rectified Attention for Efficient MLLMs30 Jun 2026
- 2026WeGenBench: A Multidimensional Diagnostic Benchmark towards Text-to-Image Model Optimization18 Jun 2026
- 2026Show the Signal, Hide the Noise: Spectral Forcing for Pixel-Space Diffusion13 Jun 2026
- 2026Representation Forcing for Bottleneck-Free Unified Multimodal Models29 May 2026
- 2026How and What to Imagine? Visual Thinking in Unified Multimodal Models for Cross-View Spatial Reasoning26 May 2026
- 2026Toward Native Multimodal Modeling: A Roadmap25 May 2026
- 2026ERA: Entropy-Guided Visual Token Pruning with Rectified Attention for Efficient MLLMscited by 2
- 2026How Far Are We from Removing the Visual Encoder? Scaling Laws for Encoder-Free Multimodal Pretrainingcited by 0
- 2026PixelUMM: Encoder-Free Unified Image and Video Understanding and Generationcited by 0
- 2026Self-correction Optimization for Interleaved Multimodal Generationcited by 0
- 2026Self-correction Optimization for Interleaved Multimodal Generation7 Oct 2026
- 2026PixelUMM: Encoder-Free Unified Image and Video Understanding and Generation29 Sep 2026
- 2026How Far Are We from Removing the Visual Encoder? Scaling Laws for Encoder-Free Multimodal Pretraining28 Sep 2026
- 2026ERA: Entropy-Guided Visual Token Pruning with Rectified Attention for Efficient MLLMs30 Jun 2026
- 2026VBVR-Pro: A Scalable and Verifiable Suite for Native Visual Reasoningcited by 4
- 2026How and What to Imagine? Visual Thinking in Unified Multimodal Models for Cross-View Spatial Reasoningcited by 2
- 2026Beyond Generation and Accuracy: Diagnosing and Enhancing Visual Chain-of-Thought for Geometry Problem Solvingcited by 0
- 2026Beyond Generation and Accuracy: Diagnosing and Enhancing Visual Chain-of-Thought for Geometry Problem Solving11 Sep 2026
- 2026VBVR-Pro: A Scalable and Verifiable Suite for Native Visual Reasoning26 Aug 2026
- 2026How and What to Imagine? Visual Thinking in Unified Multimodal Models for Cross-View Spatial Reasoning26 May 2026
- 2026Toward Native Multimodal Modeling: A Roadmapcited by 6
- 2026Toward Native Multimodal Modeling: A Roadmap25 May 2026
- 2026Vision as Unified Multimodal Generationcited by 5
- 2026Representation Forcing for Bottleneck-Free Unified Multimodal Modelscited by 4
- 2026Uncovering Understanding-Generation Synergy in Native Unified Multimodal Models: From Representation, Task to Systemcited by 2
- 2026GEAR: Guided End-to-End AutoRegression for Image Synthesiscited by 1
- 2026WeGenBench: A Multidimensional Diagnostic Benchmark towards Text-to-Image Model Optimizationcited by 0
- 2026DisciplineGen-1M: A Large-Scale Dataset for Multidisciplinary Visual Generation and Editingcited by 0
- 2026Cross-Branch Conflict as a Shield: Safeguarding Facial Identities in Unified Multimodal Image Editingcited by 0
- 2026STBridge: Shared-Target Alignment for Bridging Understanding and Generation in UMMscited by 0
- 2026Test-Time Curriculum for Open-Set AIGC Detectioncited by 0
- 2026UniWorld-Design: From Pixel Generation to Layer-Native Designcited by 0
- 2026UniSpace: Unified Visual Representation and Scalable Multimodal Modelingcited by 0
- 2026WithEveryone: Unified Planning and Identity Grounding for Group Image Generationcited by 0
12 most cited of 17 All 17 (JSON)
- 2026Reasoning-Informed Visual Editing8 Oct 2026
- 2026From Prompting to Composing: A Spatial Canvas Interface for Poster Generation8 Oct 2026
- 2026GameHorizon Suite: Multi-Horizon Data and Evaluation in Gameplay21 Sep 2026
- 2026FLAT: Resampling Image and Text into 1D Flexible-Length Aligned Transmodal Tokens for Retrieval and Generation15 Sep 2026
- 2026Editable Visual Design3 Sep 2026
- 2026Uncovering Understanding-Generation Synergy in Native Unified Multimodal Models: From Representation, Task to System1 Sep 2026
- 2026WithEveryone: Unified Planning and Identity Grounding for Group Image Generation20 Aug 2026
- 2026UniSpace: Unified Visual Representation and Scalable Multimodal Modeling9 Aug 2026
- 2026UniWorld-Design: From Pixel Generation to Layer-Native Design4 Aug 2026
- 2026Test-Time Curriculum for Open-Set AIGC Detection1 Aug 2026
- 2026STBridge: Shared-Target Alignment for Bridging Understanding and Generation in UMMs19 Jul 2026
- 2026Cross-Branch Conflict as a Shield: Safeguarding Facial Identities in Unified Multimodal Image Editing18 Jul 2026
12 newest of 17 All 17 (JSON)
Similar, not cited
Close in wording, no citation either way
- 2026From Structure to Synergy: A Survey of Vision-Language Perception Paradigm Evolution in Multimodal Large Language Models
- 2026UniG2U-Bench: Do Unified Models Advance Multimodal Understanding?
- 2025From Perception to Cognition: A Survey of Vision-Language Interactive Reasoning in Multimodal Large Language Models
- 2026A Model-Internal Protocol for Assessing Multimodal Models as Integrated Systems
- 2026Can Unified Generation and Understanding Models Maintain Semantic Equivalence Across Different Output Modalities?
- 2025A Unified Multi-Agent Framework for Universal Multimodal Understanding and Generation
- 2026UniDDT: Unifying Multimodal Understanding and Generation with Decoupled Diffusion Transformer
- 2026Reversing the Flow: Generation-to-Understanding Synergy in Large Multimodal Models