mlsanityData to 7 Oct 2026

DenseMLLM: Standard Multimodal LLMs for Dense Prediction

arXiv 2602.14134 ↗First posted 15 Feb 2026cs.CV cs.AI cs.LGcited by 0

On the map: Multimodal LLMs / vision-language models → MLLM training and visual instruction tuning

Abstract (arXiv)

Multimodal Large Language Models (MLLMs) have demonstrated exceptional capabilities in high-level visual understanding. However, extending these models to fine-grained dense prediction tasks, such as semantic segmentation and depth estimation, typically necessitates the incorporation of complex, task-specific decoders and other customizations. This architectural fragmentation increases model complexity and deviates from the generalist design of MLLMs, ultimately limiting their practicality. In this work, we challenge this paradigm by accommodating standard MLLMs to perform dense predictions without requiring additional task-specific decoders. The proposed model is called DenseMLLM, grounded in the standard architecture with a novel vision token supervision strategy for multiple labels and tasks. Despite its minimalist design, our model achieves highly competitive performance across a wide range of dense prediction and vision-language benchmarks, demonstrating that a standard, general-purpose MLLM can effectively support dense perception without architectural specialization. This project is available at github.com/Eli-YiLi/DenseMLLM.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 39 of its 48 references that are in the map. First posted on arXiv 15 Feb 2026.

Builds on

39 of 48 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025Qwen3-VL Technical Report26 Nov
  2. 2025Visual Representation Alignment for Multimodal Large Language Models9 Sep
  3. 2025BASIC: Boosting Visual Alignment with Intrinsic Refined Embeddings in Multimodal Large Language Models9 Aug
  4. 2024SEA: Supervised Embedding Alignment for Token-Level Visual-Textual Integration in MLLMs21 Aug
  5. 2024VisionLLM v2: An End-to-End Generalist Multimodal Large Language Model for Hundreds of Vision-Language Tasks12 Jun

Other references · 34

  1. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  2. 2024Qwen2.5 Technical Reportcited by 4,713
  3. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  4. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
  5. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,923
  6. 2016Pyramid Scene Parsing Networkcited by 2,472
  7. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,338
  8. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,582
  9. 2024Depth Anything V2cited by 1,385
  10. 2016Modeling Context in Referring Expressionscited by 1,054
  11. 2016COCO-Stuff: Thing and Stuff Classes in Contextcited by 858
  12. 2025SAM 3: Segment Anything with Conceptscited by 835
22 more
  1. 2021Extract Free Dense Labels from CLIPcited by 580
  2. 2016Training Region-based Object Detectors with Online Hard Example Miningcited by 577
  3. 2023VisionLLM: Large Language Model is also an Open-Ended Decoder for Vision-Centric Taskscited by 499
  4. 2023GLaMM: Pixel Grounding Large Multimodal Modelcited by 447
  5. 20193D Packing for Self-Supervised Monocular Depth Estimationcited by 437
  6. 2023Side Adapter Network for Open-Vocabulary Semantic Segmentationcited by 296
  7. 2022Generalized Decoding for Pixel, Image, and Languagecited by 292
  8. 2023Unleashing Text-to-Image Diffusion Models for Visual Perceptioncited by 266
  9. 2025UniDepthV2: Universal Monocular Metric Depth Estimation Made Simplercited by 170
  10. 20234M: Massively Multimodal Masked Modelingcited by 118
  11. 2023A Closer Look at the Explainability of Contrastive Language-Image Pre-trainingcited by 95
  12. 2025Defeating the Training-Inference Mismatch via FP16cited by 62
  13. 2023Jack of All Tasks, Master of Many: Designing General-purpose Coarse-to-Fine Vision-Language Modelcited by 52
  14. 2025Token Activation Map to Visually Explain Multimodal LLMscited by 30
  15. 2025DepthLM: Metric Depth From Vision Language Modelscited by 29
  16. 2025UFO: A Unified Approach to Fine-grained Visual Perception via Open-ended Language Interfacecited by 27
  17. 2025UniPixel: Unified Object Referring and Segmentation for Pixel-Level Visual Reasoningcited by 27
  18. 2024GiT: Towards Generalist Vision Transformer through Universal Language Interfacecited by 23
  19. 2024Distilling Spectral Graph for Object-Context Aware Open-Vocabulary Semantic Segmentationcited by 23
  20. 2024SwinMTL: A Shared Architecture for Simultaneous Depth Estimation and Semantic Segmentation from Monocular Camera Imagescited by 13
  21. 2025Youtu-LLM: Unlocking the Native Agentic Potential for Lightweight Large Language Modelscited by 8
  22. 2024Multi-modal Auto-regressive Modeling via Visual Wordscited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Dense Connector for MLLMs
  2. 2025Segmentation as A Plug-and-Play Capability for Frozen Multimodal LLMs
  3. 2024DenseFusion-1M: Merging Vision Experts for Comprehensive Multimodal Perception
  4. 2026MoR-MLLM: Mixture of Recursions for Efficient Multimodal Large Language Models
  5. 2024EE-MLLM: A Data-Efficient and Compute-Efficient Multimodal Large Language Model
  6. 2024MM1: Methods, Analysis & Insights from Multimodal LLM Pre-training
  7. 2025Towards General Continuous Memory for Vision-Language Models
  8. 2026Towards Efficient Large Vision-Language Models: A Comprehensive Survey on Inference Strategies