mlsanityData to 14 Oct 2026

Universal Instance Perception as Object Discovery and Retrieval

arXiv 2303.06674 ↗First posted 12 Mar 2023cs.CVcited by 192

Not on the map. Why?

Cited by 36 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

All instance perception tasks aim at finding certain objects specified by some queries such as category names, language expressions, and target annotations, but this complete field has been split into multiple independent subtasks. In this work, we present a universal instance perception model of the next generation, termed UNINEXT. UNINEXT reformulates diverse instance perception tasks into a unified object discovery and retrieval paradigm and can flexibly perceive different types of objects by simply changing the input prompts. This unified formulation brings the following benefits: (1) enormous data from different tasks and label vocabularies can be exploited for jointly training general instance-level representations, which is especially beneficial for tasks lacking in training data. (2) the unified model is parameter-efficient and can save redundant computation when handling multiple tasks simultaneously. UNINEXT shows superior performance on 20 challenging benchmarks from 10 instance-level tasks including classical image-level tasks (object detection and instance segmentation), vision-and-language tasks (referring expression comprehension and segmentation), and six video-level object tracking tasks. Code is available at https://github.com/MasterBin-IIAU/UNINEXT.

Key facts. Cited by 192 papers in the map (data through 14 Oct 2026). Builds on 122 of its 140 references that are in the map. First posted on arXiv 12 Mar 2023.

Builds on

122 of 140 references.

References · 122

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  5. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  6. 2020End-to-End Object Detection with Transformerscited by 7,414
  7. 2017Mask R-CNNcited by 7,277
  8. 2016Feature Pyramid Networks for Object Detectioncited by 5,036
  9. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,852
  10. 2022A ConvNet for the 2020scited by 3,579
  11. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,422
  12. 2015Fast R-CNNcited by 2,933
38 more
  1. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,351
  2. 2016V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentationcited by 2,292
  3. 2019FCOS: Fully Convolutional One-Stage Object Detectioncited by 1,577
  4. 2019Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regressioncited by 1,368
  5. 2022DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detectioncited by 1,319
  6. 2018BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learningcited by 1,288
  7. 2019LVIS: A Dataset for Large Vocabulary Instance Segmentationcited by 1,217
  8. 2021Grounded Language-Image Pre-trainingcited by 1,156
  9. 2021YOLOX: Exceeding YOLO Series in 2021cited by 1,071
  10. 2016Modeling Context in Referring Expressionscited by 1,063
  11. 2017The 2017 DAVIS Challenge on Video Object Segmentationcited by 957
  12. 2017Simple Online and Realtime Tracking with a Deep Association Metriccited by 910
  13. 2015Generation and Comprehension of Unambiguous Object Descriptionscited by 904
  14. 2021ByteTrack: Multi-Object Tracking by Associating Every Detection Boxcited by 823
  15. 2021MDETR -- Modulated Detection for End-to-End Multi-Modal Understandingcited by 819
  16. 2022A Generalist Agentcited by 808
  17. 2018Path Aggregation Network for Instance Segmentationcited by 798
  18. 2017Focal Loss for Dense Object Detectioncited by 752
  19. 2022OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Frameworkcited by 745
  20. 2022Exploring Plain Vision Transformer Backbones for Object Detectioncited by 724
  21. 2022Detecting Twenty-thousand Classes using Image-level Supervisioncited by 614
  22. 2016Fully-Convolutional Siamese Networks for Object Trackingcited by 568
  23. 2019Bridging the Gap Between Anchor-based and Anchor-free Detection via Adaptive Training Sample Selectioncited by 540
  24. 2016MOT16: A Benchmark for Multi-Object Trackingcited by 532
  25. 2021TrackFormer: Multi-Object Tracking with Transformerscited by 518
  26. 2020Sparse R-CNN: End-to-End Object Detection with Learnable Proposalscited by 495
  27. 2020FairMOT: On the Fairness of Detection and Re-Identification in Multiple Object Trackingcited by 490
  28. 2022DN-DETR: Accelerate DETR Training by Introducing Query DeNoisingcited by 488
  29. 2020Tracking Objects as Pointscited by 483
  30. 2019Hybrid Task Cascade for Instance Segmentationcited by 481
  31. 2018GOT-10k: A Large High-Diversity Benchmark for Generic Object Tracking in the Wildcited by 475
  32. 2018SiamRPN++: Evolution of Siamese Visual Tracking with Very Deep Networkscited by 446
  33. 2019YOLACT: Real-time Instance Segmentationcited by 443
  34. 2020End-to-End Video Instance Segmentation with Transformerscited by 443
  35. 2019PointRend: Image Segmentation as Renderingcited by 440
  36. 2022XMem: Long-Term Video Object Segmentation with an Atkinson-Shiffrin Memory Modelcited by 439
  37. 2019Video Object Segmentation using Space-Time Memory Networkscited by 438
  38. 2018LaSOT: A High-quality Benchmark for Large-scale Single Object Trackingcited by 429

50 most cited of 122 All 122 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 46; 2024: 66; 2025: 60; 2026: 20.

192 papers.

Sort:

By area:

  1. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
  2. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,510
  3. 2024Grounded SAM: Assembling Open-World Models for Diverse Visual Taskscited by 929
  4. 2023Shikra: Unleashing Multimodal LLM's Referential Dialogue Magiccited by 906
  5. 2023MiniGPT-v2: large language model as a unified interface for vision-language multi-task learningcited by 625
  6. 2023Segment Everything Everywhere All at Oncecited by 548
  7. 2023VisionLLM: Large Language Model is also an Open-Ended Decoder for Vision-Centric Taskscited by 501
  8. 2023Florence-2: Advancing a Unified Representation for a Variety of Vision Taskscited by 410
  9. 2023Unified-IO 2: Scaling Autoregressive Multimodal Models with Vision, Language, Audio, and Actioncited by 302
  10. 2023SPHINX: The Joint Mixing of Weights, Tasks, and Visual Embeddings for Multi-modal Large Language Modelscited by 286
  11. 2024Visual CoT: Advancing Multi-Modal Language Models with a Comprehensive Dataset and Benchmark for Chain-of-Thought Reasoningcited by 280
  12. 2023Tracking Anything with Decoupled Video Segmentationcited by 197
38 more
  1. 2023Putting the Object Back into Video Object Segmentationcited by 188
  2. 2025Sa2VA: Marrying SAM2 with MLLM for Dense Grounded Understanding of Images and Videoscited by 159
  3. 2024VisionLLM v2: An End-to-End Generalist Multimodal Large Language Model for Hundreds of Vision-Language Taskscited by 137
  4. 2024OMG-LLaVA: Bridging Image-level, Object-level, Pixel-level Reasoning and Understandingcited by 127
  5. 2024Groma: Localized Visual Tokenization for Grounding Multimodal Large Language Modelscited by 119
  6. 2023LLaVA-Grounding: Grounded Visual Chat with Large Multimodal Modelscited by 113
  7. 2024PSALM: Pixelwise SegmentAtion with Large Multi-Modal Modelcited by 101
  8. 2025UniVG-R1: Reasoning Guided Universal Visual Grounding with Reinforcement Learningcited by 84
  9. 2024MoVA: Adapting Mixture of Vision Experts to Multimodal Contextcited by 83
  10. 2023Fine-Grained Visual Promptingcited by 82
  11. 2024OMG-Seg: Is One Model Good Enough For All Segmentation?cited by 75
  12. 2023General Object Foundation Model for Images and Videos at Scalecited by 71
  13. 2024GROUNDHOG: Grounding Large Language Models to Holistic Segmentationcited by 70
  14. 2023ChatVideo: A Tracklet-centric Multimodal and Versatile Video Understanding Systemcited by 65
  15. 2023Single-Model and Any-Modality for Video Object Trackingcited by 65
  16. 2024OneTracker: Unifying Visual Object Tracking with Foundation Models and Efficient Tuningcited by 65
  17. 2024EVF-SAM: Early Vision-Language Fusion for Text-Prompted Segment Anything Modelcited by 65
  18. 2023OnlineRefer: A Simple Online Baseline for Referring Video Object Segmentationcited by 62
  19. 2024Text4Seg: Reimagining Image Segmentation as Text Generationcited by 61
  20. 2023Aligning and Prompting Everything All at Once for Universal Visual Perceptioncited by 60
  21. 2023Referred by Multi-Modality: A Unified Temporal Transformer for Video Object Segmentationcited by 54
  22. 2023Hierarchical Open-vocabulary Universal Image Segmentationcited by 54
  23. 2023Visual In-Context Promptingcited by 52
  24. 2024Toward Visual Grounding: A Surveycited by 52
  25. 2024Matching Anything by Segmenting Anythingcited by 50
  26. 2023CoDet: Co-Occurrence Guided Region-Word Alignment for Open-Vocabulary Object Detectioncited by 46
  27. 2024DetToolChain: A New Prompting Paradigm to Unleash Detection Ability of MLLMcited by 46
  28. 2023GREC: Generalized Referring Expression Comprehensioncited by 42
  29. 2023Described Object Detection: Liberating Object Detection with Flexible Expressionscited by 40
  30. 2024Multi-modal Instruction Tuned LLMs with Fine-grained Visual Perceptioncited by 39
  31. 2025VisRL: Intention-Driven Visual Perception via Reinforced Reasoningcited by 39
  32. 2024SimVG: A Simple Framework for Visual Grounding with Decoupled Multi-modal Fusioncited by 36
  33. 2024LocCa: Visual Pretraining with Location-aware Captionerscited by 35
  34. 2025MeViS: A Multi-Modal Dataset for Referring Motion Expression Video Segmentationcited by 35
  35. 2023Ref-Diff: Zero-shot Referring Image Segmentation with Generative Modelscited by 33
  36. 2024TextHawk2: A Large Vision-Language Model Excels in Bilingual OCR and Grounding with 16x Fewer Tokenscited by 32
  37. 2023Griffon: Spelling out All Object Locations at Any Granularity with Large Language Modelscited by 31
  38. 2025X-SAM: From Segment Anything to Any Segmentationcited by 30

50 most cited of 192 All 192 (JSON)