mlsanityData to 7 Oct 2026

LongCat-Next: Lexicalizing Modalities as Discrete Tokens

arXiv 2603.27538 ↗First posted 29 Mar 2026cs.CV cs.CLcited by 30

Not on the map. Why?

Abstract (arXiv)

The prevailing Next-Token Prediction (NTP) paradigm has driven the success of large language models through discrete autoregressive modeling. However, contemporary multimodal systems remain language-centric, often treating non-linguistic modalities as external attachments, leading to fragmented architectures and suboptimal integration. To transcend this limitation, we introduce Discrete Native Autoregressive (DiNA), a unified framework that represents multimodal information within a shared discrete space, enabling a consistent and principled autoregressive modeling across modalities. A key innovation is the Discrete Native Any-resolution Visual Transformer (dNaViT), which performs tokenization and de-tokenization at arbitrary resolutions, transforming continuous visual signals into hierarchical discrete tokens. Building on this foundation, we develop LongCat-Next, a native multimodal model that processes text, vision, and audio under a single autoregressive objective with minimal modality-specific design. As an industrial-strength foundation model, it excels at seeing, painting, and talking within a single framework, achieving strong performance across a wide range of multimodal benchmarks. In particular, LongCat-Next addresses the long-standing performance ceiling of discrete vision modeling on understanding tasks and provides a unified approach to effectively reconcile the conflict between understanding and generation. As an attempt toward native multimodality, we open-source the LongCat-Next and its tokenizers, hoping to foster further research and development in the community. GitHub: https://github.com/meituan-longcat/LongCat-Next

Key facts. Cited by 30 papers in the map (data through 7 Oct 2026). Builds on 123 of its 141 references that are in the map. First posted on arXiv 29 Mar 2026.

Builds on

123 of 141 references.

References · 123

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  4. 2025Qwen3 Technical Reportcited by 8,195
  5. 2023Visual Instruction Tuningcited by 7,792
  6. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  7. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  8. 2022Flow Matching for Generative Modelingcited by 5,426
  9. 2017Neural Discrete Representation Learningcited by 4,650
  10. 2024GPT-4o System Cardcited by 4,467
  11. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  12. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
38 more
  1. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
  2. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  3. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
  4. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
  5. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
  6. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,551
  7. 2025Qwen3-VL Technical Reportcited by 2,534
  8. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,105
  9. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
  10. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,758
  11. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,582
  12. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,449
  13. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,288
  14. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,139
  15. 2023Kosmos-2: Grounding Multimodal Large Language Models to the Worldcited by 1,075
  16. 2016Modeling Context in Referring Expressionscited by 1,054
  17. 2025Qwen-Image Technical Reportcited by 973
  18. 2025FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Spacecited by 960
  19. 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignmentcited by 946
  20. 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 904
  21. 2024Are We on the Right Way for Evaluating Large Vision-Language Models?cited by 893
  22. 2025Emerging Properties in Unified Multimodal Pretrainingcited by 855
  23. 2023Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesiscited by 847
  24. 2016A Diagram Is Worth A Dozen Imagescited by 793
  25. 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 787
  26. 2024Emu3: Next-Token Prediction is All You Needcited by 761
  27. 2022Autoregressive Image Generation using Residual Quantizationcited by 706
  28. 2023C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Modelscited by 694
  29. 2023DataComp: In search of the next generation of multimodal datasetscited by 635
  30. 2025Flow-GRPO: Training Flow Matching Models via Online RLcited by 605
  31. 2025UI-TARS: Pioneering Automated GUI Interaction with Native Agentscited by 584
  32. 2024Moshi: a speech-text foundation model for real-time dialoguecited by 536
  33. 2024Measuring Multimodal Mathematical Reasoning with MATH-Vision Datasetcited by 516
  34. 2026Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfacescited by 515
  35. 2024MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmarkcited by 494
  36. 2023OCRBench: On the Hidden Mystery of OCR in Large Multimodal Modelscited by 466
  37. 2022FLEURS: Few-shot Learning Evaluation of Universal Representations of Speechcited by 458
  38. 2024ELLA: Equip Diffusion Models with LLM for Enhanced Semantic Alignmentcited by 455

50 most cited of 123 All 123 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 30.

30 papers.

Sort:

By area:

  1. 2026ImageWAM: Do World Action Models Really Need Video Generation, or Just Image Editing?cited by 33
  2. 2026SenseNova-U1: Unifying Multimodal Understanding and Generation with NEO-unify Architecturecited by 27
  3. 2026VGGT-Omegacited by 20
  4. 2026GenClaw: Code-Driven Agentic Image Generationcited by 10
  5. 2026Toward Native Multimodal Modeling: A Roadmapcited by 6
  6. 2026Visual Generation in the New Era: An Evolution from Atomic Mapping to Agentic World Modelingcited by 5
  7. 2026EvoStreaming: Your Offline Video Model Is a Natively Streaming Assistantcited by 5
  8. 2026OScaR: The Occam's Razor for Extreme KV Cache Quantization in LLMs and Beyondcited by 5
  9. 2026Ask, Solve, Generate: Self-Evolving Unified Multimodal Understanding and Generation via Self-Consistency Rewardscited by 5
  10. 2026AutoDesign: Meta-Harness Optimization for Long-Horizon Agentic Designcited by 5
  11. 2026Unveiling Fine-Grained Visual Traces: Evaluating Multimodal Interleaved Reasoning Chains in Multimodal STEM Taskscited by 4
  12. 2026Uni-Edit: Intelligent Editing Is A General Task For Unified Model Tuningcited by 4
18 more
  1. 2026Beyond the Last Layer: Multi-Layer Representation Fusion for Visual Tokenizationcited by 3
  2. 2026PhyWorld: Physics-Faithful World Model for Video Generationcited by 3
  3. 2026OmniInteract: Benchmarking Real-World Streaming Interaction for Real-Time Omnimodal Assistantscited by 3
  4. 2026SenseNova-U1.5: Towards Native Unified Visual Intelligencecited by 3
  5. 2026InterleaveThinker: Reinforcing Agentic Interleaved Generationcited by 2
  6. 2026Thinking Once Is Enough: Intermediate-Layer Evidence Routing for High-Resolution VQAcited by 1
  7. 2026Towards Physics of Multimodal Pretraining: Knowledge Flow, Modality Synergy, Early Unification, and Recipescited by 1
  8. 2026Massive Activations in Hybrid Linear Attention Large Language Models: Pre-Attention Spikes and Inter-Spike Plateauscited by 1
  9. 2026Generation as Auxiliary Supervision: Enhancing Visual Understanding at Zero Inference Overhead via Decoupled Embedding Predictioncited by 1
  10. 2026FireRedAudio: A General-Purpose Audio Language Model with Decoupled Continuous Representations for Understanding and Generationcited by 1
  11. 2026Nemotron-Labs-Diffusion-Image: Advancing Masked Discrete Diffusion for High-Resolution Image Synthesiscited by 0
  12. 2026Omni-Flow: A Unified Workflow Orchestration and Distributed KV Cache Sharing Framework for Multimodal Inferencecited by 0
  13. 2026dRAE: Representation Autoencoder with Hyper-Spherical Codescited by 0
  14. 2026When Does Visual Generation Help Visual Understanding in Unified Multimodal Models?cited by 0
  15. 2026Studying Image Tokenizers as Visual Languages in Unified Multimodal Modelscited by 0
  16. 2026Reducing the Output-Mode Gap in Speech Language Models via Joint-Output On-Policy Distillationcited by 0
  17. 2026PixelUMM: Encoder-Free Unified Image and Video Understanding and Generationcited by 0
  18. 2026DC-SAE: Deep Compression Semantic Autoencoder for Faster Diffusion Convergencecited by 0