mlsanityData to 7 Oct 2026

Moondream Segmentation: From Words to Masks

arXiv 2604.02593 ↗First posted 3 Apr 2026cs.CV cs.AIcited by 0

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

Abstract (arXiv)

We present Moondream Segmentation, a referring image segmentation extension of Moondream 3, a vision-language model. Given an image and a referring expression, the model autoregressively decodes a vector path and iteratively refines the rasterized mask into a final detailed mask. We introduce a reinforcement learning stage that resolves ambiguity in the supervised signal by directly optimizing mask quality. Rollouts from this stage produce coarse-to-ground-truth targets for the refiner. To mitigate evaluation noise from polygon annotations, we release RefCOCO-M, a cleaned RefCOCO validation split with boundary-accurate masks. Moondream Segmentation achieves a cIoU of 80.2% on RefCOCO (val) and 62.6% mIoU on LVIS (val).

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 21 of its 28 references that are in the map. First posted on arXiv 3 Apr 2026.

Builds on

21 of 28 references.

References · 21

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  4. 2023Segment Anythingcited by 8,158
  5. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  6. 2024SAM 2: Segment Anything in Images and Videoscited by 2,784
  7. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,338
  8. 2016V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentationcited by 2,284
  9. 2020Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domainscited by 2,170
  10. 2019LVIS: A Dataset for Large Vocabulary Instance Segmentationcited by 1,215
  11. 2025SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Featurescited by 1,127
  12. 2016Modeling Context in Referring Expressionscited by 1,054
9 more
  1. 2015Generation and Comprehension of Unambiguous Object Descriptionscited by 897
  2. 2024PaliGemma: A versatile 3B VLM for transfercited by 679
  3. 2023Segment Anything in High Qualitycited by 373
  4. 2021Pix2seq: A Language Modeling Framework for Object Detectioncited by 345
  5. 2016Segmentation from Natural Language Expressionscited by 257
  6. 2021Boundary IoU: Improving Object-Centric Image Segmentation Evaluationcited by 186
  7. 2023PolyFormer: Referring Image Segmentation as Sequential Polygon Generationcited by 147
  8. 2018Efficient Interactive Annotation of Segmentation Datasets with Polygon-RNN++cited by 119
  9. 2024Text4Seg: Reimagining Image Segmentation as Text Generationcited by 59

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Refer to Any Segmentation Mask Group With Vision-Language Prompts
  2. 2026B-GRTO: Bootstrapped Group Relative Tool Optimization for Referring Segmentation
  3. 2025LENS: Learning to Segment Anything with Unified Reinforced Reasoning
  4. 2026Qwen3-VL-Seg: Unlocking Open-World Referring Segmentation with Vision-Language Grounding
  5. 2025Guideline-Consistent Segmentation via Multi-Agent Refinement
  6. 2025R2SM: Referring and Reasoning for Selective Masks
  7. 2026VespaSeg: A Resource-Aware Ground-then-Segment Pipeline for Referring Expression Segmentation ↗
  8. 2024MaskDiffusion: Exploiting Pre-trained Diffusion Models for Semantic Segmentation