NExT-OMNI: Towards Any-to-Any Omnimodal Foundation Models with Discrete Flow Matching
On the map: Diffusion models → Image generation, text-to-image and editing
Abstract (arXiv)
Next-generation multimodal foundation models capable of any-to-any cross-modal generation and multi-turn interaction will serve as core components of artificial general intelligence systems, playing a pivotal role in human-machine interaction. However, most existing multimodal models remain constrained by autoregressive architectures, whose inherent limitations prevent a balanced integration of understanding and generation capabilities. Although hybrid and decoupling strategies have been explored to address these tasks within unified frameworks separately, their redundant, non-integrated designs limit their applicability to broader scenarios, such as cross-modal retrieval. In this work, we introduce NExT-OMNI, an open-source omnimodal foundation model that achieves unified modeling through discrete flow paradigms. By leveraging metric-induced probability paths and kinetic optimal velocities, NExT-OMNI natively supports any-to-any understanding and generation with enhanced response efficiency, while enabling broader application scenarios through concise unified representations rather than task-decoupled designs. Trained on large-scale interleaved text, image, video, and audio data, NExT-OMNI delivers competitive performance on multimodal generation and understanding benchmarks, while outperforming prior unified models in multi-turn multimodal interaction and cross-modal retrieval, highlighting its architectural advantages as a next-generation multimodal foundation model. To advance further research, we release training details, data protocols, and open-source both the code and model checkpoints.
Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 120 of its 146 references that are in the map. First posted on arXiv 15 Oct 2025.
Builds on
120 of 146 references.
Closest ancestors · 11
Same direction, newest first
- 2025FUDOKI: Discrete Flow-based Unified Understanding and Generation via Kinetic-Optimal Velocities26 May
- 2025MMaDA: Multimodal Large Diffusion Language Models21 May
- 2025BLIP3-o: A Family of Fully Open Unified Multimodal Models-Architecture, Training and Dataset14 May
- 2025MMGen: Unified Multi-modal Image Generation and Understanding in One Go26 Mar
- 2024Dual Diffusion for Unified Image Generation and Understanding31 Dec
- 2024Emu3: Next-Token Prediction is All You Need27 Sep
- 2024VILA-U: a Unified Foundation Model Integrating Visual Understanding and Generation6 Sep
- 2024ANOLE: An Open, Autoregressive, Native Large Multimodal Models for Interleaved Image-Text Generation8 Jul
- 2024DEEM: Diffusion Models Serve as the Eyes of Large Language Models for Image Perception24 May
- 2024Playground v2.5: Three Insights towards Enhancing Aesthetic Quality in Text-to-Image Generation27 Feb
- 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignment17 Oct
Other references · 109
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
- 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
- 2023Visual Instruction Tuningcited by 7,792
- 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
- 2022Classifier-Free Diffusion Guidancecited by 5,717
- 2022Flow Matching for Generative Modelingcited by 5,426
- 2025Qwen2.5-VL Technical Reportcited by 5,418
- 2024Qwen2.5 Technical Reportcited by 4,713
- 2017Neural Discrete Representation Learningcited by 4,650
- 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
38 more
- 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
- 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
- 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
- 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
- 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
- 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,498
- 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,105
- 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
- 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,755
- 2023Evaluating Object Hallucination in Large Vision-Language Modelscited by 1,696
- 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,624
- 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understandingcited by 1,461
- 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,385
- 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,311
- 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,221
- 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,116
- 2023Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Modelscited by 1,047
- 2023ShareGPT4V: Improving Large Multi-Modal Models with Better Captionscited by 1,000
- 2022CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformerscited by 973
- 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 904
- 2025Large Language Diffusion Modelscited by 872
- 2025Emerging Properties in Unified Multimodal Pretrainingcited by 855
- 2023VideoChat: Chat-Centric Video Understandingcited by 851
- 2017GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networkscited by 816
- 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 787
- 2024VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMscited by 749
- 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 734
- 2023NExT-GPT: Any-to-Any Multimodal LLMcited by 610
- 2023Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Modelscited by 591
- 2024DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understandingcited by 559
- 2024LLaVA-Video: Video Instruction Tuning With Synthetic Datacited by 550
- 2024Moshi: a speech-text foundation model for real-time dialoguecited by 536
- 2023SpeechGPT: Empowering Large Language Models with Intrinsic Cross-Modal Conversational Abilitiescited by 518
- 2025Dream 7B: Diffusion Large Language Modelscited by 515
- 2024Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Modelcited by 468
- 2024NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Modelscited by 466
- 2024Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generationcited by 447
- 2023Generative Multimodal Models are In-Context Learnerscited by 446
50 most cited of 109 All 109 (JSON)
Built on by
Citing papers per year: 2025: 3; 2026: 16.
19 papers.
Sort:
By area:
- 2025TUNA: Taming Unified Visual Representations for Native Unified Multimodal Modelscited by 34
- 2026OmniGAIA: Towards Native Omni-Modal AI Agentscited by 21
- 2026UniDriveVLA: Unifying Understanding, Perception, and Action Planning for Autonomous Drivingcited by 17
- 2026Omni-Diffusion: Unified Multimodal Understanding and Generation with Masked Discrete Diffusioncited by 9
- 2025OmniGuard: Unified Omni-Modal Guardrails with Deliberate Reasoningcited by 5
- 2026OmniFysics: Towards Physical Intelligence Evolution via Omni-Modal Signal Processing and Network Optimizationcited by 5
- 2026BATQuant: Outlier-resilient MXFP4 Quantization via Learnable Block-wise Optimizationcited by 5
- 2026DFM-VLA: Iterative Action Refinement for Robot Manipulation via Discrete Flow Matchingcited by 5
- 2026FreeAct: Freeing Activations for LLM Quantizationcited by 3
- 2026UDM-GRPO: Stable and Efficient Group Relative Policy Optimization for Uniform Discrete Diffusion Modelscited by 3
- 2025ITS3D: Inference-Time Scaling for Text-Guided 3D Diffusion Modelscited by 1
- 2026Generalizable Multimodal Large Language Model Editing via Invariant Trajectory Learningcited by 1
7 more
- 2026TraceAV-Bench: Benchmarking Multi-Hop Trajectory Reasoning over Long Audio-Visual Videoscited by 1
- 2026Kinetic-Optimal Scheduling with Moment Correction for Metric-Induced Discrete Flow Matching in Zero-Shot Text-to-Speechcited by 1
- 2026Omnimodal Dataset Distillation via High-order Proxy Alignmentcited by 0
- 2026A Time-Reparameterized Cumulative Intensity Extrapolation Sampler for Discrete Flow Matchingcited by 0
- 2026Representation Learning in Diffusion and Flow-based Model: An Application Aspectcited by 0
- 2026Can MiniMax-H3 Reason About the Physical World? An Evaluation of Omni-Modal Generative Modelcited by 0
- 2026PixelUMM: Encoder-Free Unified Image and Video Understanding and Generationcited by 0
- 2026PixelUMM: Encoder-Free Unified Image and Video Understanding and Generation29 Sep 2026
- 2026Can MiniMax-H3 Reason About the Physical World? An Evaluation of Omni-Modal Generative Model16 Sep 2026
- 2026Representation Learning in Diffusion and Flow-based Model: An Application Aspect25 Aug 2026
- 2026A Time-Reparameterized Cumulative Intensity Extrapolation Sampler for Discrete Flow Matching23 Jun 2026
- 2026Kinetic-Optimal Scheduling with Moment Correction for Metric-Induced Discrete Flow Matching in Zero-Shot Text-to-Speech10 May 2026
- 2026TraceAV-Bench: Benchmarking Multi-Hop Trajectory Reasoning over Long Audio-Visual Videos8 May 2026
- 2026UDM-GRPO: Stable and Efficient Group Relative Policy Optimization for Uniform Discrete Diffusion Models20 Apr 2026
- 2026Omnimodal Dataset Distillation via High-order Proxy Alignment12 Apr 2026
- 2026UniDriveVLA: Unifying Understanding, Perception, and Action Planning for Autonomous Driving2 Apr 2026
- 2026DFM-VLA: Iterative Action Refinement for Robot Manipulation via Discrete Flow Matching27 Mar 2026
- 2026BATQuant: Outlier-resilient MXFP4 Quantization via Learnable Block-wise Optimization17 Mar 2026
- 2026Omni-Diffusion: Unified Multimodal Understanding and Generation with Masked Discrete Diffusion6 Mar 2026
7 more
- 2026FreeAct: Freeing Activations for LLM Quantization2 Mar 2026
- 2026OmniGAIA: Towards Native Omni-Modal AI Agents26 Feb 2026
- 2026OmniFysics: Towards Physical Intelligence Evolution via Omni-Modal Signal Processing and Network Optimization5 Feb 2026
- 2026Generalizable Multimodal Large Language Model Editing via Invariant Trajectory Learning27 Jan 2026
- 2025OmniGuard: Unified Omni-Modal Guardrails with Deliberate Reasoning2 Dec 2025
- 2025TUNA: Taming Unified Visual Representations for Native Unified Multimodal Models1 Dec 2025
- 2025ITS3D: Inference-Time Scaling for Text-Guided 3D Diffusion Models27 Nov 2025
- 2026Kinetic-Optimal Scheduling with Moment Correction for Metric-Induced Discrete Flow Matching in Zero-Shot Text-to-Speechcited by 1
- 2026A Time-Reparameterized Cumulative Intensity Extrapolation Sampler for Discrete Flow Matchingcited by 0
- 2026Representation Learning in Diffusion and Flow-based Model: An Application Aspectcited by 0
- 2026Representation Learning in Diffusion and Flow-based Model: An Application Aspect25 Aug 2026
- 2026A Time-Reparameterized Cumulative Intensity Extrapolation Sampler for Discrete Flow Matching23 Jun 2026
- 2026Kinetic-Optimal Scheduling with Moment Correction for Metric-Induced Discrete Flow Matching in Zero-Shot Text-to-Speech10 May 2026
- 2026PixelUMM: Encoder-Free Unified Image and Video Understanding and Generation29 Sep 2026
- 2026Omni-Diffusion: Unified Multimodal Understanding and Generation with Masked Discrete Diffusion6 Mar 2026
- 2026Generalizable Multimodal Large Language Model Editing via Invariant Trajectory Learning27 Jan 2026
- 2026OmniGAIA: Towards Native Omni-Modal AI Agentscited by 21
- 2026OmniGAIA: Towards Native Omni-Modal AI Agents26 Feb 2026
- 2025TUNA: Taming Unified Visual Representations for Native Unified Multimodal Modelscited by 34
- 2026OmniFysics: Towards Physical Intelligence Evolution via Omni-Modal Signal Processing and Network Optimizationcited by 5
- 2026BATQuant: Outlier-resilient MXFP4 Quantization via Learnable Block-wise Optimizationcited by 5
- 2026FreeAct: Freeing Activations for LLM Quantizationcited by 3
- 2026TraceAV-Bench: Benchmarking Multi-Hop Trajectory Reasoning over Long Audio-Visual Videoscited by 1
- 2026Omnimodal Dataset Distillation via High-order Proxy Alignmentcited by 0
- 2026Can MiniMax-H3 Reason About the Physical World? An Evaluation of Omni-Modal Generative Modelcited by 0
- 2026Can MiniMax-H3 Reason About the Physical World? An Evaluation of Omni-Modal Generative Model16 Sep 2026
- 2026TraceAV-Bench: Benchmarking Multi-Hop Trajectory Reasoning over Long Audio-Visual Videos8 May 2026
- 2026Omnimodal Dataset Distillation via High-order Proxy Alignment12 Apr 2026
- 2026BATQuant: Outlier-resilient MXFP4 Quantization via Learnable Block-wise Optimization17 Mar 2026
- 2026FreeAct: Freeing Activations for LLM Quantization2 Mar 2026
- 2026OmniFysics: Towards Physical Intelligence Evolution via Omni-Modal Signal Processing and Network Optimization5 Feb 2026
- 2025TUNA: Taming Unified Visual Representations for Native Unified Multimodal Models1 Dec 2025
Similar, not cited
Close in wording, no citation either way
- 2025Uni-MoE-2.0-Omni: Scaling Language-Centric Omnimodal Large Model with Advanced MoE, Training and Data
- 2026MiniCPM-o 4.5: Towards Real-Time Full-Duplex Omni-Modal Interaction
- 2026Dynin-Omni: Omnimodal Unified Large Diffusion Language Model
- 2025M2-omni: Advancing Omni-MLLM for Comprehensive Modality Support with Competitive Performance
- 2026AR-Omni: A Unified Autoregressive Model for Any-to-Any Generation
- 2025OmniMamba: Efficient and Unified Multimodal Understanding and Generation via State Space Models
- 2026LLaDA-o: An Effective and Length-Adaptive Omni Diffusion Model
- 2026Omni-DuplexEval: Evaluating Real-time Duplex Omni-modal Interaction