mlsanityData to 14 Oct 2026

OMG-Seg: Is One Model Good Enough For All Segmentation?

arXiv 2401.10229 ↗First posted 18 Jan 2024cs.CVcited by 75

Not on the map. Why?

Abstract (arXiv)

In this work, we address various segmentation tasks, each traditionally tackled by distinct or partially unified models. We propose OMG-Seg, One Model that is Good enough to efficiently and effectively handle all the segmentation tasks, including image semantic, instance, and panoptic segmentation, as well as their video counterparts, open vocabulary settings, prompt-driven, interactive segmentation like SAM, and video object segmentation. To our knowledge, this is the first model to handle all these tasks in one model and achieve satisfactory performance. We show that OMG-Seg, a transformer-based encoder-decoder architecture with task-specific queries and outputs, can support over ten distinct segmentation tasks and yet significantly reduce computational and parameter overhead across various tasks and datasets. We rigorously evaluate the inter-task influences and correlations during co-training. Code and models are available at https://github.com/lxtGH/OMG-Seg.

Key facts. Cited by 75 papers in the map (data through 14 Oct 2026). Builds on 96 of its 112 references that are in the map. First posted on arXiv 18 Jan 2024.

Builds on

96 of 112 references.

References · 96

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  5. 2023Segment Anythingcited by 8,234
  6. 2020End-to-End Object Detection with Transformerscited by 7,414
  7. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  8. 2017Mask R-CNNcited by 7,277
  9. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  10. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
  11. 2022A ConvNet for the 2020scited by 3,579
  12. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,422
38 more
  1. 2018Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentationcited by 3,223
  2. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,351
  3. 2016V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentationcited by 2,292
  4. 2017Rethinking Atrous Convolution for Semantic Image Segmentationcited by 2,161
  5. 2021TransUNet: Transformers Make Strong Encoders for Medical Image Segmentationcited by 1,558
  6. 2019MMDetection: Open MMLab Detection Toolbox and Benchmarkcited by 1,385
  7. 2021Per-Pixel Classification is Not All You Need for Semantic Segmentationcited by 1,152
  8. 2023Kosmos-2: Grounding Multimodal Large Language Models to the Worldcited by 1,089
  9. 2016Semantic Understanding of Scenes through the ADE20K Datasetcited by 1,069
  10. 2021Open-vocabulary Object Detection via Vision and Language Knowledge Distillationcited by 906
  11. 2015Generation and Comprehension of Unambiguous Object Descriptionscited by 904
  12. 2018Panoptic Segmentationcited by 802
  13. 2022Language-driven Semantic Segmentationcited by 717
  14. 2022EVA: Exploring the Limits of Masked Visual Representation Learning at Scalecited by 691
  15. 2020Image Segmentation Using Deep Learning: A Surveycited by 660
  16. 2019Panoptic Feature Pyramid Networkscited by 645
  17. 2022Detecting Twenty-thousand Classes using Image-level Supervisioncited by 614
  18. 2022Vision Transformer Adapter for Dense Predictionscited by 565
  19. 2023Segment Everything Everywhere All at Oncecited by 548
  20. 2023VisionLLM: Large Language Model is also an Open-Ended Decoder for Vision-Centric Taskscited by 501
  21. 2019Hybrid Task Cascade for Instance Segmentationcited by 481
  22. 2021Scaling Open-Vocabulary Image Segmentation with Image-Level Labelscited by 473
  23. 2020End-to-End Video Instance Segmentation with Transformerscited by 443
  24. 2019Video Object Segmentation using Space-Time Memory Networkscited by 438
  25. 2020Open-Vocabulary Object Detection Using Captionscited by 427
  26. 2019Video Instance Segmentationcited by 393
  27. 2023Open-Vocabulary Panoptic Segmentation with Text-to-Image Diffusion Modelscited by 375
  28. 2019Panoptic-DeepLabcited by 363
  29. 2020MaX-DeepLab: End-to-End Panoptic Segmentation with Mask Transformerscited by 355
  30. 2021Pix2seq: A Language Modeling Framework for Object Detectioncited by 349
  31. 2022OneFormer: One Transformer to Rule Universal Image Segmentationcited by 328
  32. 2022Images Speak in Images: A Generalist Painter for In-Context Visual Learningcited by 318
  33. 2022Generalized Decoding for Pixel, Image, and Languagecited by 294
  34. 2019SOLO: Segmenting Objects by Locationscited by 289
  35. 2022SegNeXt: Rethinking Convolutional Attention Design for Semantic Segmentationcited by 252
  36. 2022Visual Prompting via Image Inpaintingcited by 250
  37. 2023MOSE: A New Dataset for Video Object Segmentation in Complex Scenescited by 234
  38. 2023MeViS: A Large-scale Benchmark for Video Segmentation with Motion Expressionscited by 228

50 most cited of 96 All 96 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 26; 2025: 34; 2026: 15.

75 papers.

Sort:

By area:

  1. 2025Sa2VA: Marrying SAM2 with MLLM for Dense Grounded Understanding of Images and Videoscited by 159
  2. 2024OMG-LLaVA: Bridging Image-level, Object-level, Pixel-level Reasoning and Understandingcited by 127
  3. 2024PSALM: Pixelwise SegmentAtion with Large Multi-Modal Modelcited by 101
  4. 2024SAM2-UNet: Segment Anything 2 Makes Strong Encoder for Natural and Medical Image Segmentationcited by 64
  5. 2025Your ViT is Secretly an Image Segmentation Modelcited by 63
  6. 2025MOSEv2: A More Challenging Dataset for Video Object Segmentation in Complex Scenescited by 62
  7. 2024Vitron: A Unified Pixel-level Vision LLM for Understanding, Generating, Segmenting, Editingcited by 49
  8. 2024Towards Semantic Equivalence of Tokenization in Multimodal LLMcited by 45
  9. 2024Graphic Design with Large Multimodal Modelcited by 40
  10. 2024Image Segmentation in Foundation Model Era: A Surveycited by 35
  11. 2024FaceXFormer: A Unified Transformer for Facial Analysiscited by 34
  12. 2024Self-Calibrated CLIP for Training-Free Open-Vocabulary Segmentationcited by 34
38 more
  1. 2024HyperSeg: Towards Universal Visual Segmentation with Large Language Modelcited by 33
  2. 2025X-SAM: From Segment Anything to Any Segmentationcited by 30
  3. 2024ObjectRelator: Enabling Cross-View Object Relation Understanding Across Ego-Centric and Exo-Centric Perspectivescited by 29
  4. 2025From Perception to Cognition: A Survey of Vision-Language Interactive Reasoning in Multimodal Large Language Modelscited by 26
  5. 2024Stepping Stones: A Progressive Training Strategy for Audio-Visual Semantic Segmentationcited by 23
  6. 2024CapHuman: Capture Your Moments in Parallel Universescited by 22
  7. 2024MG-LLaVA: Towards Multi-Granularity Visual Instruction Tuningcited by 22
  8. 2025Pixel-SAIL: Single Transformer For Pixel-Grounded Understandingcited by 22
  9. 2025Multimodal Referring Segmentation: A Surveycited by 22
  10. 2024DGMamba: Domain Generalization via Generalized State Space Modelcited by 21
  11. 2024SemFlow: Binding Semantic Segmentation and Image Synthesis via Rectified Flowcited by 20
  12. 2024LoRACLR: Contrastive Adaptation for Customization of Diffusion Modelscited by 20
  13. 2025HiMTok: Learning Hierarchical Mask Tokens for Image Segmentation with Large Multimodal Modelcited by 20
  14. 2025PanSt3R: Multi-view Consistent Panoptic Segmentationcited by 20
  15. 2025Are They the Same? Exploring Visual Correspondence Shortcomings of Multimodal LLMscited by 16
  16. 2025DenseWorld-1M: Towards Detailed Dense Grounded Caption in the Real Worldcited by 16
  17. 2024A Unified Framework for 3D Scene Understandingcited by 15
  18. 2024Explore In-Context Segmentation via Latent Diffusion Modelscited by 12
  19. 2024Clustering Propagation for Universal Medical Image Segmentationcited by 12
  20. 2025Universal Scene Graph Generationcited by 11
  21. 2024PosSAM: Panoptic Open-vocabulary Segment Anythingcited by 9
  22. 2024GenView: Enhancing View Quality with Pretrained Generative Model for Self-Supervised Learningcited by 9
  23. 2025Leader360V: The Large-scale, Real-world 360 Video Dataset for Multi-task Learning in Diverse Environmentcited by 8
  24. 2025O-MaMa: Learning Object Mask Matching between Egocentric and Exocentric Viewscited by 7
  25. 2025The 1st Solution for 7th LSVOS RVOS Track: SaSaSa2VAcited by 7
  26. 2025Unified Open-World Segmentation with Multi-Modal Promptscited by 7
  27. 2025Learning 4D Panoptic Scene Graph Generation from Rich 2D Visual Scenecited by 6
  28. 2025TextSplat: Text-Guided Semantic Fusion for Generalizable Gaussian Splattingcited by 6
  29. 2024Open-Vocabulary Action Localization with Iterative Visual Promptingcited by 5
  30. 2025Cross-View Multi-Modal Segmentation @ Ego-Exo4D Challenges 2025cited by 5
  31. 2025LIRA: Inferring Segmentation in Large Multi-modal Models with Local Interleaved Region Assistancecited by 5
  32. 2025Complementary and Contrastive Learning for Audio-Visual Segmentationcited by 5
  33. 2026MMLoP: Multi-Modal Low-Rank Prompting for Efficient Vision-Language Adaptationcited by 5
  34. 2025SpaceSeg: A High-Precision Intelligent Perception Segmentation Method for Multi-Spacecraft On-Orbit Targetscited by 3
  35. 2025Beyond Appearance: Geometric Cues for Robust Video Instance Segmentationcited by 3
  36. 2025Distribution-Specific Learning for Joint Salient and Camouflaged Object Detectioncited by 3
  37. 2025Human-in-Context: Unified Cross-Domain 3D Human Motion Modeling via In-Context Learningcited by 3
  38. 2025VaMP: Variational Multi-Modal Prompt Learning for Vision-Language Modelscited by 3

50 most cited of 75 All 75 (JSON)