LongCat-Next: Lexicalizing Modalities as Discrete Tokens
Not on the map. Why?
Abstract (arXiv)
The prevailing Next-Token Prediction (NTP) paradigm has driven the success of large language models through discrete autoregressive modeling. However, contemporary multimodal systems remain language-centric, often treating non-linguistic modalities as external attachments, leading to fragmented architectures and suboptimal integration. To transcend this limitation, we introduce Discrete Native Autoregressive (DiNA), a unified framework that represents multimodal information within a shared discrete space, enabling a consistent and principled autoregressive modeling across modalities. A key innovation is the Discrete Native Any-resolution Visual Transformer (dNaViT), which performs tokenization and de-tokenization at arbitrary resolutions, transforming continuous visual signals into hierarchical discrete tokens. Building on this foundation, we develop LongCat-Next, a native multimodal model that processes text, vision, and audio under a single autoregressive objective with minimal modality-specific design. As an industrial-strength foundation model, it excels at seeing, painting, and talking within a single framework, achieving strong performance across a wide range of multimodal benchmarks. In particular, LongCat-Next addresses the long-standing performance ceiling of discrete vision modeling on understanding tasks and provides a unified approach to effectively reconcile the conflict between understanding and generation. As an attempt toward native multimodality, we open-source the LongCat-Next and its tokenizers, hoping to foster further research and development in the community. GitHub: https://github.com/meituan-longcat/LongCat-Next
Key facts. Cited by 30 papers in the map (data through 7 Oct 2026). Builds on 123 of its 141 references that are in the map. First posted on arXiv 29 Mar 2026.
Builds on
123 of 141 references.
References · 123
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
- 2025Qwen3 Technical Reportcited by 8,195
- 2023Visual Instruction Tuningcited by 7,792
- 2020Measuring Massive Multitask Language Understandingcited by 7,768
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
- 2022Flow Matching for Generative Modelingcited by 5,426
- 2017Neural Discrete Representation Learningcited by 4,650
- 2024GPT-4o System Cardcited by 4,467
- 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
- 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
38 more
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
- 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
- 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
- 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
- 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
- 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,551
- 2025Qwen3-VL Technical Reportcited by 2,534
- 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,105
- 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
- 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,758
- 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,582
- 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,449
- 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,288
- 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,139
- 2023Kosmos-2: Grounding Multimodal Large Language Models to the Worldcited by 1,075
- 2016Modeling Context in Referring Expressionscited by 1,054
- 2025Qwen-Image Technical Reportcited by 973
- 2025FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Spacecited by 960
- 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignmentcited by 946
- 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 904
- 2024Are We on the Right Way for Evaluating Large Vision-Language Models?cited by 893
- 2025Emerging Properties in Unified Multimodal Pretrainingcited by 855
- 2023Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesiscited by 847
- 2016A Diagram Is Worth A Dozen Imagescited by 793
- 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 787
- 2024Emu3: Next-Token Prediction is All You Needcited by 761
- 2022Autoregressive Image Generation using Residual Quantizationcited by 706
- 2023C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Modelscited by 694
- 2023DataComp: In search of the next generation of multimodal datasetscited by 635
- 2025Flow-GRPO: Training Flow Matching Models via Online RLcited by 605
- 2025UI-TARS: Pioneering Automated GUI Interaction with Native Agentscited by 584
- 2024Moshi: a speech-text foundation model for real-time dialoguecited by 536
- 2024Measuring Multimodal Mathematical Reasoning with MATH-Vision Datasetcited by 516
- 2026Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfacescited by 515
- 2024MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmarkcited by 494
- 2023OCRBench: On the Hidden Mystery of OCR in Large Multimodal Modelscited by 466
- 2022FLEURS: Few-shot Learning Evaluation of Universal Representations of Speechcited by 458
- 2024ELLA: Equip Diffusion Models with LLM for Enhanced Semantic Alignmentcited by 455
50 most cited of 123 All 123 (JSON)
Built on by
Citing papers per year: 2026: 30.
30 papers.
Sort:
By area:
- 2026ImageWAM: Do World Action Models Really Need Video Generation, or Just Image Editing?cited by 33
- 2026SenseNova-U1: Unifying Multimodal Understanding and Generation with NEO-unify Architecturecited by 27
- 2026VGGT-Omegacited by 20
- 2026GenClaw: Code-Driven Agentic Image Generationcited by 10
- 2026Toward Native Multimodal Modeling: A Roadmapcited by 6
- 2026Visual Generation in the New Era: An Evolution from Atomic Mapping to Agentic World Modelingcited by 5
- 2026EvoStreaming: Your Offline Video Model Is a Natively Streaming Assistantcited by 5
- 2026OScaR: The Occam's Razor for Extreme KV Cache Quantization in LLMs and Beyondcited by 5
- 2026Ask, Solve, Generate: Self-Evolving Unified Multimodal Understanding and Generation via Self-Consistency Rewardscited by 5
- 2026AutoDesign: Meta-Harness Optimization for Long-Horizon Agentic Designcited by 5
- 2026Unveiling Fine-Grained Visual Traces: Evaluating Multimodal Interleaved Reasoning Chains in Multimodal STEM Taskscited by 4
- 2026Uni-Edit: Intelligent Editing Is A General Task For Unified Model Tuningcited by 4
18 more
- 2026Beyond the Last Layer: Multi-Layer Representation Fusion for Visual Tokenizationcited by 3
- 2026PhyWorld: Physics-Faithful World Model for Video Generationcited by 3
- 2026OmniInteract: Benchmarking Real-World Streaming Interaction for Real-Time Omnimodal Assistantscited by 3
- 2026SenseNova-U1.5: Towards Native Unified Visual Intelligencecited by 3
- 2026InterleaveThinker: Reinforcing Agentic Interleaved Generationcited by 2
- 2026Thinking Once Is Enough: Intermediate-Layer Evidence Routing for High-Resolution VQAcited by 1
- 2026Towards Physics of Multimodal Pretraining: Knowledge Flow, Modality Synergy, Early Unification, and Recipescited by 1
- 2026Massive Activations in Hybrid Linear Attention Large Language Models: Pre-Attention Spikes and Inter-Spike Plateauscited by 1
- 2026Generation as Auxiliary Supervision: Enhancing Visual Understanding at Zero Inference Overhead via Decoupled Embedding Predictioncited by 1
- 2026FireRedAudio: A General-Purpose Audio Language Model with Decoupled Continuous Representations for Understanding and Generationcited by 1
- 2026Nemotron-Labs-Diffusion-Image: Advancing Masked Discrete Diffusion for High-Resolution Image Synthesiscited by 0
- 2026Omni-Flow: A Unified Workflow Orchestration and Distributed KV Cache Sharing Framework for Multimodal Inferencecited by 0
- 2026dRAE: Representation Autoencoder with Hyper-Spherical Codescited by 0
- 2026When Does Visual Generation Help Visual Understanding in Unified Multimodal Models?cited by 0
- 2026Studying Image Tokenizers as Visual Languages in Unified Multimodal Modelscited by 0
- 2026Reducing the Output-Mode Gap in Speech Language Models via Joint-Output On-Policy Distillationcited by 0
- 2026PixelUMM: Encoder-Free Unified Image and Video Understanding and Generationcited by 0
- 2026DC-SAE: Deep Compression Semantic Autoencoder for Faster Diffusion Convergencecited by 0
- 2026DC-SAE: Deep Compression Semantic Autoencoder for Faster Diffusion Convergence30 Sep 2026
- 2026PixelUMM: Encoder-Free Unified Image and Video Understanding and Generation29 Sep 2026
- 2026Reducing the Output-Mode Gap in Speech Language Models via Joint-Output On-Policy Distillation14 Sep 2026
- 2026SenseNova-U1.5: Towards Native Unified Visual Intelligence10 Sep 2026
- 2026Studying Image Tokenizers as Visual Languages in Unified Multimodal Models8 Sep 2026
- 2026FireRedAudio: A General-Purpose Audio Language Model with Decoupled Continuous Representations for Understanding and Generation25 Aug 2026
- 2026When Does Visual Generation Help Visual Understanding in Unified Multimodal Models?23 Aug 2026
- 2026AutoDesign: Meta-Harness Optimization for Long-Horizon Agentic Design13 Aug 2026
- 2026Generation as Auxiliary Supervision: Enhancing Visual Understanding at Zero Inference Overhead via Decoupled Embedding Prediction12 Aug 2026
- 2026Massive Activations in Hybrid Linear Attention Large Language Models: Pre-Attention Spikes and Inter-Spike Plateaus12 Aug 2026
- 2026Towards Physics of Multimodal Pretraining: Knowledge Flow, Modality Synergy, Early Unification, and Recipes5 Aug 2026
- 2026Thinking Once Is Enough: Intermediate-Layer Evidence Routing for High-Resolution VQA30 Jul 2026
18 more
- 2026dRAE: Representation Autoencoder with Hyper-Spherical Codes24 Jul 2026
- 2026Omni-Flow: A Unified Workflow Orchestration and Distributed KV Cache Sharing Framework for Multimodal Inference30 Jun 2026
- 2026Nemotron-Labs-Diffusion-Image: Advancing Masked Discrete Diffusion for High-Resolution Image Synthesis29 Jun 2026
- 2026Ask, Solve, Generate: Self-Evolving Unified Multimodal Understanding and Generation via Self-Consistency Rewards25 Jun 2026
- 2026ImageWAM: Do World Action Models Really Need Video Generation, or Just Image Editing?17 Jun 2026
- 2026InterleaveThinker: Reinforcing Agentic Interleaved Generation11 Jun 2026
- 2026GenClaw: Code-Driven Agentic Image Generation28 May 2026
- 2026OmniInteract: Benchmarking Real-World Streaming Interaction for Real-Time Omnimodal Assistants26 May 2026
- 2026Toward Native Multimodal Modeling: A Roadmap25 May 2026
- 2026Uni-Edit: Intelligent Editing Is A General Task For Unified Model Tuning20 May 2026
- 2026OScaR: The Occam's Razor for Extreme KV Cache Quantization in LLMs and Beyond19 May 2026
- 2026PhyWorld: Physics-Faithful World Model for Video Generation19 May 2026
- 2026VGGT-Omega14 May 2026
- 2026SenseNova-U1: Unifying Multimodal Understanding and Generation with NEO-unify Architecture12 May 2026
- 2026Beyond the Last Layer: Multi-Layer Representation Fusion for Visual Tokenization11 May 2026
- 2026EvoStreaming: Your Offline Video Model Is a Natively Streaming Assistant11 May 2026
- 2026Visual Generation in the New Era: An Evolution from Atomic Mapping to Agentic World Modeling30 Apr 2026
- 2026Unveiling Fine-Grained Visual Traces: Evaluating Multimodal Interleaved Reasoning Chains in Multimodal STEM Tasks21 Apr 2026
- 2026ImageWAM: Do World Action Models Really Need Video Generation, or Just Image Editing?cited by 33
- 2026Toward Native Multimodal Modeling: A Roadmapcited by 6
- 2026Visual Generation in the New Era: An Evolution from Atomic Mapping to Agentic World Modelingcited by 5
- 2026PhyWorld: Physics-Faithful World Model for Video Generationcited by 3
- 2026ImageWAM: Do World Action Models Really Need Video Generation, or Just Image Editing?17 Jun 2026
- 2026Toward Native Multimodal Modeling: A Roadmap25 May 2026
- 2026PhyWorld: Physics-Faithful World Model for Video Generation19 May 2026
- 2026Visual Generation in the New Era: An Evolution from Atomic Mapping to Agentic World Modeling30 Apr 2026
- 2026AutoDesign: Meta-Harness Optimization for Long-Horizon Agentic Design13 Aug 2026
- 2026InterleaveThinker: Reinforcing Agentic Interleaved Generation11 Jun 2026
- 2026GenClaw: Code-Driven Agentic Image Generation28 May 2026
- 2026Ask, Solve, Generate: Self-Evolving Unified Multimodal Understanding and Generation via Self-Consistency Rewardscited by 5
- 2026Unveiling Fine-Grained Visual Traces: Evaluating Multimodal Interleaved Reasoning Chains in Multimodal STEM Taskscited by 4
- 2026Thinking Once Is Enough: Intermediate-Layer Evidence Routing for High-Resolution VQAcited by 1
- 2026Thinking Once Is Enough: Intermediate-Layer Evidence Routing for High-Resolution VQA30 Jul 2026
- 2026Ask, Solve, Generate: Self-Evolving Unified Multimodal Understanding and Generation via Self-Consistency Rewards25 Jun 2026
- 2026Unveiling Fine-Grained Visual Traces: Evaluating Multimodal Interleaved Reasoning Chains in Multimodal STEM Tasks21 Apr 2026
- 2026SenseNova-U1: Unifying Multimodal Understanding and Generation with NEO-unify Architecturecited by 27
- 2026Generation as Auxiliary Supervision: Enhancing Visual Understanding at Zero Inference Overhead via Decoupled Embedding Predictioncited by 1
- 2026PixelUMM: Encoder-Free Unified Image and Video Understanding and Generationcited by 0
- 2026PixelUMM: Encoder-Free Unified Image and Video Understanding and Generation29 Sep 2026
- 2026Generation as Auxiliary Supervision: Enhancing Visual Understanding at Zero Inference Overhead via Decoupled Embedding Prediction12 Aug 2026
- 2026SenseNova-U1: Unifying Multimodal Understanding and Generation with NEO-unify Architecture12 May 2026
- 2026VGGT-Omegacited by 20
- 2026EvoStreaming: Your Offline Video Model Is a Natively Streaming Assistantcited by 5
- 2026Uni-Edit: Intelligent Editing Is A General Task For Unified Model Tuningcited by 4
- 2026Beyond the Last Layer: Multi-Layer Representation Fusion for Visual Tokenizationcited by 3
- 2026OmniInteract: Benchmarking Real-World Streaming Interaction for Real-Time Omnimodal Assistantscited by 3
- 2026SenseNova-U1.5: Towards Native Unified Visual Intelligencecited by 3
- 2026Towards Physics of Multimodal Pretraining: Knowledge Flow, Modality Synergy, Early Unification, and Recipescited by 1
- 2026Massive Activations in Hybrid Linear Attention Large Language Models: Pre-Attention Spikes and Inter-Spike Plateauscited by 1
- 2026FireRedAudio: A General-Purpose Audio Language Model with Decoupled Continuous Representations for Understanding and Generationcited by 1
- 2026dRAE: Representation Autoencoder with Hyper-Spherical Codescited by 0
- 2026When Does Visual Generation Help Visual Understanding in Unified Multimodal Models?cited by 0
- 2026Studying Image Tokenizers as Visual Languages in Unified Multimodal Modelscited by 0
12 most cited of 13 All 13 (JSON)
- 2026Reducing the Output-Mode Gap in Speech Language Models via Joint-Output On-Policy Distillation14 Sep 2026
- 2026SenseNova-U1.5: Towards Native Unified Visual Intelligence10 Sep 2026
- 2026Studying Image Tokenizers as Visual Languages in Unified Multimodal Models8 Sep 2026
- 2026FireRedAudio: A General-Purpose Audio Language Model with Decoupled Continuous Representations for Understanding and Generation25 Aug 2026
- 2026When Does Visual Generation Help Visual Understanding in Unified Multimodal Models?23 Aug 2026
- 2026Massive Activations in Hybrid Linear Attention Large Language Models: Pre-Attention Spikes and Inter-Spike Plateaus12 Aug 2026
- 2026Towards Physics of Multimodal Pretraining: Knowledge Flow, Modality Synergy, Early Unification, and Recipes5 Aug 2026
- 2026dRAE: Representation Autoencoder with Hyper-Spherical Codes24 Jul 2026
- 2026OmniInteract: Benchmarking Real-World Streaming Interaction for Real-Time Omnimodal Assistants26 May 2026
- 2026Uni-Edit: Intelligent Editing Is A General Task For Unified Model Tuning20 May 2026
- 2026VGGT-Omega14 May 2026
- 2026Beyond the Last Layer: Multi-Layer Representation Fusion for Visual Tokenization11 May 2026
12 newest of 13 All 13 (JSON)