mlsanityData to 14 Oct 2026

Qwen3.5-Omni Technical Report

arXiv 2604.15804 ↗First posted 17 Apr 2026cs.CL eess.AScited by 174

On the map: Mixture-of-experts → Multimodal MoE models

Abstract (arXiv)

In this work, we present Qwen3.5-Omni, the latest advancement in the Qwen-Omni model family. Representing a significant evolution over its predecessor, Qwen3.5-Omni scales to hundreds of billions of parameters and supports a 256k context length. By leveraging a massive dataset comprising heterogeneous text-vision pairs and over 100 million hours of audio-visual content, the model demonstrates robust omni-modality capabilities. Qwen3.5-Omni-plus achieves SOTA results across 215 audio and audio-visual understanding, reasoning, and interaction subtasks and benchmarks, surpassing Gemini-3.1 Pro in key audio tasks and matching it in comprehensive audio-visual understanding. Architecturally, Qwen3.5-Omni employs a Hybrid Attention Mixture-of-Experts (MoE) framework for both Thinker and Talker, enabling efficient long-sequence inference. The model facilitates sophisticated interaction, supporting over 10 hours of audio understanding and 400 seconds of 720P video (at 1 FPS). To address the inherent instability and unnaturalness in streaming speech synthesis, often caused by encoding efficiency discrepancies between text and speech tokenizers, we introduce ARIA. ARIA dynamically aligns text and speech units, significantly enhancing the stability and prosody of conversational speech with minimal latency impact. Furthermore, Qwen3.5-Omni expands linguistic boundaries, supporting multilingual understanding and speech generation across 10 languages with human-like emotional nuance. Finally, Qwen3.5-Omni exhibits superior audio-visual grounding capabilities, generating script-level structured captions with precise temporal synchronization and automated scene segmentation. Remarkably, we observed the emergence of a new capability in omnimodal models: directly performing coding based on audio-visual instructions, which we call Audio-Visual Vibe Coding.

Key facts. Cited by 174 papers in the map (data through 14 Oct 2026). Placed in Multimodal MoE models, one of 12 directions in Mixture-of-experts. Builds on 79 of its 96 references that are in the map. First posted on arXiv 17 Apr 2026.

Builds on

79 of 96 references.

References · 79

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  3. 2025Qwen3 Technical Reportcited by 8,820
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  5. 2023Visual Instruction Tuningcited by 7,869
  6. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  7. 2025Qwen2.5-VL Technical Reportcited by 5,587
  8. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  9. 2023Qwen Technical Reportcited by 3,344
  10. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  11. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
  12. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
38 more
  1. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
  2. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 2,004
  3. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,848
  4. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,614
  5. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,413
  6. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,384
  7. 2023Instruction-Following Evaluation for Large Language Modelscited by 1,228
  8. 2021Grounded Language-Image Pre-trainingcited by 1,156
  9. 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmarkcited by 1,109
  10. 2024Are We on the Right Way for Evaluating Large Vision-Language Models?cited by 914
  11. 2025Group Sequence Policy Optimizationcited by 835
  12. 2016A Diagram Is Worth A Dozen Imagescited by 804
  13. 2025Qwen2.5-Omni Technical Reportcited by 736
  14. 2023C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Modelscited by 708
  15. 2025τ2-Bench: Evaluating Conversational Agents in a Dual-Control Environmentcited by 627
  16. 2023Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Modelscited by 608
  17. 2023HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Modelscited by 583
  18. 2024Qwen2-Audio Technical Reportcited by 560
  19. 2024Measuring Multimodal Mathematical Reasoning with MATH-Vision Datasetcited by 526
  20. 2024MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmarkcited by 512
  21. 2025Qwen3-Omni Technical Reportcited by 497
  22. 2023OCRBench: On the Hidden Mystery of OCR in Large Multimodal Modelscited by 476
  23. 2022FLEURS: Few-shot Learning Evaluation of Universal Representations of Speechcited by 469
  24. 2021SLAKE: A Semantically-Labeled Knowledge-Enhanced Dataset for Medical Visual Question Answeringcited by 449
  25. 2024LVBench: An Extreme Long Video Understanding Benchmarkcited by 410
  26. 2024LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitaskscited by 379
  27. 2024F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matchingcited by 361
  28. 2023PMC-VQA: Visual Instruction Tuning for Medical Visual Question Answeringcited by 338
  29. 2024Seed-TTS: A Family of High-Quality Versatile Speech Generation Modelscited by 300
  30. 2025MathArena: Evaluating LLMs on Uncontaminated Math Competitionscited by 297
  31. 2024MLVU: Benchmarking Multi-task Long Video Understandingcited by 292
  32. 2024MME-RealWorld: Could Your Multimodal LLM Challenge High-Resolution Real-World Scenarios that are Difficult for Humans?cited by 292
  33. 2024CosyVoice 2: Scalable Streaming Speech Synthesis with Large Language Modelscited by 288
  34. 2024CharXiv: Charting Gaps in Realistic Chart Understanding in Multimodal LLMscited by 234
  35. 2025MedXpertQA: Benchmarking Expert-Level Medical Reasoning and Understandingcited by 224
  36. 2024VoiceBench: Benchmarking LLM-Based Voice Assistantscited by 200
  37. 2023Teaching CLIP to Count to Tencited by 198
  38. 2021WenetSpeech: A 10000+ Hours Multi-domain Mandarin Corpus for Speech Recognitioncited by 197

50 most cited of 79 All 79 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 174.

174 papers.

Sort:

By area:

  1. 2026Gemma 4 Technical Reportcited by 259
  2. 2026Cosmos 3: Omnimodal World Models for Physical AIcited by 165
  3. 2026WorldArena 2.0: Extending Embodied World Model Benchmarking on Modality, Functionality and Platformcited by 10
  4. 2026nuReasoning: A Reasoning-Centric Dataset and Benchmark for Long-Tail Autonomous Drivingcited by 9
  5. 2026Do Joint Audio-Video Generation Models Understand Physics?cited by 8
  6. 2026IPIBench: Evaluating Interactive Proactive Intelligence of MLLMs under Continuous Streamscited by 7
  7. 2026Spatial-Omni: Spatial Audio Understanding Integration in Multimodal LLMs via FOA Encodingcited by 7
  8. 2026Learning User Simulators with Turing Rewardscited by 7
  9. 2026Wan-Streamer v0.1: End-to-end Real-time Interactive Foundation Modelscited by 7
  10. 2026Safety Testing LLM Agents at Scale: From Risk Discovery to Evidence-Grounded Verificationcited by 7
  11. 2026Neglected Free Lunch from Post-training: Progress Advantage for LLM Agentscited by 6
  12. 2026When to Think, When to Speak: Learning Disclosure Policies for LLM Reasoningcited by 5
38 more
  1. 2026From History to State: Constant-Context Skill Learning for LLM Agentscited by 5
  2. 2026Moment-Video: Diagnosing Temporal Fidelity of Video MLLMs on Momentary Visual Eventscited by 5
  3. 2026MMAE: A Massive Multitask Audio Editing Benchmarkcited by 5
  4. 2026Taming I2V models for Image HOI Editing: A Cognitive Benchmark and Agentic Self-Correcting Frameworkcited by 5
  5. 2026Attend to Your Own Thoughts: Breaking the Barrier for Post-Training Quantization of Reasoning LLMs through the Lens of 1.58-Bit Quantizationcited by 5
  6. 2026DIVE: Unlocking Self-Improvement in Frozen Language Models Through Diversity-Driven Skill Evolutioncited by 5
  7. 2026VA-Judger: Reward Modeling from Human Preference Feedback for Joint Video-Audio Generationcited by 5
  8. 2026PresentAgent-2: Towards Generalist Multimodal Presentation Agentscited by 4
  9. 2026Conditioning Gaussian Processes on Almost Anythingcited by 4
  10. 2026BraveGuard: From Open-World Threats to Safer Computer-Use Agentscited by 4
  11. 2026Video-MME-Logical: A Controlled Diagnostic Benchmark for Video Temporal-Logical Reasoningcited by 4
  12. 2026SG-WAM: Self-Guided World Modeling in Geometry-Aware Policy Spacecited by 4
  13. 2026OmniRefine: Alignment-Aware Cooperative Compression for Efficient Omnimodal Large Language Modelscited by 3
  14. 2026Stage-adaptive Token Selection for Efficient Omni-modal LLMscited by 3
  15. 2026MobileMoE: Scaling On-Device Mixture of Expertscited by 3
  16. 2026Agentic Active Omni-Modal Perception for Multi-Hop Audio-Visual Reasoningcited by 3
  17. 2026YoCausal: How Far is Video Generation from World Model? A Causality Perspectivecited by 3
  18. 2026Multi-Agent Computer Usecited by 3
  19. 2026Which Tokens Matter? Adaptive Token Selection for RLVR with the Relative Surprisal Indexcited by 3
  20. 2026GMO-E2DIT: Grounded Multi-Operation Editing for E-Commerce Imagescited by 3
  21. 2026Audio-Zero: Label-Free Self-Evolution for Fine-Grained Audio Reasoningcited by 3
  22. 2026Chimera: Designing and Chinchilla-Scaling Hybrid Visual Diffusion Transformerscited by 3
  23. 2026OmniPack: Unified Token Compression for Efficient Omni-modal Large Language Modelscited by 3
  24. 2026Multimodal Duplex Interaction Agentcited by 3
  25. 2026RetireOPD: Self-Retiring On-Policy Distillation for Agentic Reinforcement Learningcited by 3
  26. 2026Towards Generation-Efficient Uncertainty Estimation in Large Language Modelscited by 2
  27. 2026FraudBench: A Multimodal Benchmark for Detecting AI-Generated Fraudulent Refund Evidencecited by 2
  28. 2026Overcoming Catastrophic Forgetting in Visual Continual Learning with Reinforcement Fine-Tuningcited by 2
  29. 2026MM-Snowball: Evaluating and Mitigating Hallucination Snowballing in Multimodal Multi-Turn Dialoguecited by 2
  30. 2026StainFlow: Entity-Stain Tracking and Evidence Linking for Process Rewards in GUI Agentscited by 2
  31. 2026Orchestra-o1: Omnimodal Agent Orchestrationcited by 2
  32. 2026OmniFocus: Query-Guided Modality-Balanced Token Compression for Omni-Modal Large Language Modelscited by 2
  33. 2026UniClawBench: A Universal Benchmark for Proactive Agents on Real-World Taskscited by 2
  34. 2026Learning Generalizable Behaviors for Terminal Agentscited by 2
  35. 2026FireRedAudio: A General-Purpose Audio Language Model with Decoupled Continuous Representations for Understanding and Generationcited by 2
  36. 2026VoiceMem: Streaming Dual-Brain Memory for Real-Time Interactioncited by 2
  37. 2026TTPO: Test-Time Policy Optimizationcited by 2
  38. 2026Qwen3.8-Omni: Towards Native Omni-Modal Agentscited by 2

50 most cited of 174 All 174 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Uni-MoE-2.0-Omni: Scaling Language-Centric Omnimodal Large Model with Advanced MoE, Training and Data
  2. 2026Boosting Omni-Modal Language Models: Staged Post-Training with Visually Debiased Evaluation
  3. 2025Nexus: An Omni-Perceptive And -Interactive Model for Language, Audio, And Vision
  4. 2025OmniVideoBench: Towards Audio-Visual Understanding Evaluation for Omni MLLMs
  5. 2024OMCAT: Omni Context Aware Transformer
  6. 2026Training-Free Speech-Centric Omni Understanding with Frozen VLMs
  7. 2025LongCat-Flash-Omni Technical Report
  8. 2026Audio-Omni: Extending Multi-modal Understanding to Versatile Audio Generation and Editing