# High-Quality Entity Segmentation

arXiv: https://arxiv.org/abs/2211.05776 · HTML: https://mlsanity.com/paper/2211.05776/

## Key facts

- Cited by 93 papers in the map (data through 14 Oct 2026).
- Builds on 46 of its 63 references that are in the map.
- First posted on arXiv 10 Nov 2022.

## Builds on (most cited first, 30 of 46)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [Pyramid Scene Parsing Network](https://mlsanity.com/paper/1612.01105/) (2016)
- [Masked-attention Mask Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2112.01527/) (2021)
- [LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairs](https://mlsanity.com/paper/2111.02114/) (2021)
- [Deformable Convolutional Networks](https://mlsanity.com/paper/1703.06211/) (2017)
- [LVIS: A Dataset for Large Vocabulary Instance Segmentation](https://mlsanity.com/paper/1908.03195/) (2019)
- [Per-Pixel Classification is Not All You Need for Semantic Segmentation](https://mlsanity.com/paper/2107.06278/) (2021)
- [Playing for Data: Ground Truth from Computer Games](https://mlsanity.com/paper/1608.02192/) (2016)
- [Panoptic Segmentation](https://mlsanity.com/paper/1801.00868/) (2018)
- [Path Aggregation Network for Instance Segmentation](https://mlsanity.com/paper/1803.01534/) (2018)
- [Focal Loss for Dense Object Detection](https://mlsanity.com/paper/1708.02002/) (2017)
- [Panoptic Feature Pyramid Networks](https://mlsanity.com/paper/1901.02446/) (2019)
- [MaskGAN: Towards Diverse and Interactive Facial Image Manipulation](https://mlsanity.com/paper/1907.11922/) (2019)
- [DOTA: A Large-scale Dataset for Object Detection in Aerial Images](https://mlsanity.com/paper/1711.10398/) (2017)
- [The Open Images Dataset V4: Unified image classification, object detection, and visual relationship detection at scale](https://mlsanity.com/paper/1811.00982/) (2018)
- [Video Instance Segmentation](https://mlsanity.com/paper/1905.04804/) (2019)
- [MaX-DeepLab: End-to-End Panoptic Segmentation with Mask Transformers](https://mlsanity.com/paper/2012.00759/) (2020)
- [Conditional Convolutions for Instance Segmentation](https://mlsanity.com/paper/2003.05664/) (2020)
- [SOLO: Segmenting Objects by Locations](https://mlsanity.com/paper/1912.04488/) (2019)
- [Cross-Domain Weakly-Supervised Object Detection through Progressive Domain Adaptation](https://mlsanity.com/paper/1803.11365/) (2018)
- [Mask2Former for Video Instance Segmentation](https://mlsanity.com/paper/2112.10764/) (2021)
- [FSS-1000: A 1000-Class Dataset for Few-Shot Segmentation](https://mlsanity.com/paper/1907.12347/) (2019)

## Built on by (most cited first, 30 of 93)

- [Osprey: Pixel Understanding with Visual Instruction Tuning](https://mlsanity.com/paper/2312.10032/) (2023)
- [OMG-LLaVA: Bridging Image-level, Object-level, Pixel-level Reasoning and Understanding](https://mlsanity.com/paper/2406.19389/) (2024)
- [Open-Vocabulary SAM: Segment and Recognize Twenty-thousand Classes Interactively](https://mlsanity.com/paper/2401.02955/) (2024)
- [GROUNDHOG: Grounding Large Language Models to Holistic Segmentation](https://mlsanity.com/paper/2402.16846/) (2024)
- [A Survey on Open-Vocabulary Detection and Segmentation: Past, Present, and Future](https://mlsanity.com/paper/2307.09220/) (2023)
- [Gen3DSR: Generalizable 3D Scene Reconstruction via Divide and Conquer from a Single View](https://mlsanity.com/paper/2404.03421/) (2024)
- [MaskClustering: View Consensus based Mask Graph Clustering for Open-Vocabulary 3D Instance Segmentation](https://mlsanity.com/paper/2401.07745/) (2024)
- [Towards Semantic Equivalence of Tokenization in Multimodal LLM](https://mlsanity.com/paper/2406.05127/) (2024)
- [Gaga: Group Any Gaussians via 3D-aware Memory Bank](https://mlsanity.com/paper/2404.07977/) (2024)
- [DICEPTION: A Generalist Diffusion Model for Visual Perceptual Tasks](https://mlsanity.com/paper/2502.17157/) (2025)
- [SAM-guided Graph Cut for 3D Instance Segmentation](https://mlsanity.com/paper/2312.08372/) (2023)
- [CorrCLIP: Reconstructing Patch Correlations in CLIP for Open-Vocabulary Semantic Segmentation](https://mlsanity.com/paper/2411.10086/) (2024)
- [ZeroShape: Regression-based Zero-shot Shape Reconstruction](https://mlsanity.com/paper/2312.14198/) (2023)
- [UniGS: Unified Representation for Image Generation and Segmentation](https://mlsanity.com/paper/2312.01985/) (2023)
- [COCONut: Modernizing COCO Segmentation](https://mlsanity.com/paper/2404.08639/) (2024)
- [DiT360: High-Fidelity Panoramic Image Generation via Hybrid Training](https://mlsanity.com/paper/2510.11712/) (2025)
- [Rethinking Interactive Image Segmentation with Low Latency, High Quality, and Diverse Prompts](https://mlsanity.com/paper/2404.00741/) (2024)
- [AnyControl: Create Your Artwork with Versatile Control on Text-to-Image Generation](https://mlsanity.com/paper/2406.18958/) (2024)
- [Zero-Shot Object-Centric Representation Learning](https://mlsanity.com/paper/2408.09162/) (2024)
- [Region-Based Representations Revisited](https://mlsanity.com/paper/2402.02352/) (2024)
- [Mamba or RWKV: Exploring High-Quality and High-Efficiency Segment Anything Model](https://mlsanity.com/paper/2406.19369/) (2024)
- [HiScene: Creating Hierarchical 3D Scenes with Isometric View Generation](https://mlsanity.com/paper/2504.13072/) (2025)
- [PanoGS: Gaussian-based Panoptic Segmentation for 3D Open Vocabulary Scene Understanding](https://mlsanity.com/paper/2503.18107/) (2025)
- [Any3DIS: Class-Agnostic 3D Instance Segmentation by 2D Mask Tracking](https://mlsanity.com/paper/2411.16183/) (2024)
- [MonoInstance: Enhancing Monocular Priors via Multi-view Instance Alignment for Neural Rendering and Reconstruction](https://mlsanity.com/paper/2503.18363/) (2025)
- [High-Quality Entity Segmentation and Grounding](https://mlsanity.com/paper/2402.02555/) (2024)
- [Unified Dense Prediction of Video Diffusion](https://mlsanity.com/paper/2503.09344/) (2025)
- [OnlineAnySeg: Online Zero-Shot 3D Segmentation by Visual Foundation Model Guided 2D Mask Merging](https://mlsanity.com/paper/2503.01309/) (2025)
- [AirSim360: A Panoramic Simulation Platform within Drone View](https://mlsanity.com/paper/2512.02009/) (2025)
- [Are We Done with Object-Centric Learning?](https://mlsanity.com/paper/2504.07092/) (2025)

## Abstract (arXiv)

Dense image segmentation tasks e.g., semantic, panoptic) are useful for image editing, but existing methods can hardly generalize well in an in-the-wild setting where there are unrestricted image domains, classes, and image resolution and quality variations. Motivated by these observations, we construct a new entity segmentation dataset, with a strong focus on high-quality dense segmentation in the wild. The dataset contains images spanning diverse image domains and entities, along with plentiful high-resolution images and high-quality mask annotations for training and testing. Given the high-quality and -resolution nature of the dataset, we propose CropFormer which is designed to tackle the intractability of instance-level segmentation on high-resolution images. It improves mask prediction by fusing high-res image crops that provide more fine-grained image details and the full image. CropFormer is the first query-based Transformer architecture that can effectively fuse mask predictions from multiple image views, by learning queries that effectively associate the same entities across the full image and its crop. With CropFormer, we achieve a significant AP gain of 1.9 on the challenging entity segmentation task. Furthermore, CropFormer consistently improves the accuracy of traditional segmentation tasks and datasets. The dataset and code will be released at http://luqi.info/entityv2.github.io/.