# TensorMask: A Foundation for Dense Object Segmentation

arXiv: https://arxiv.org/abs/1903.12174 · HTML: https://mlsanity.com/paper/1903.12174/

## Key facts

- Cited by 101 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Mar 2019.

## Built on by (most cited first, 30 of 101)

- [K-Net: Towards Unified Image Segmentation](https://mlsanity.com/paper/2106.14855/) (2021)
- [Transformer-Based Visual Segmentation: A Survey](https://mlsanity.com/paper/2304.09854/) (2023)
- [Deep Occlusion-Aware Instance Segmentation with Overlapping BiLayers](https://mlsanity.com/paper/2103.12340/) (2021)
- [SeqFormer: Sequential Transformer for Video Instance Segmentation](https://mlsanity.com/paper/2112.08275/) (2021)
- [Crossover Learning for Fast Online Video Instance Segmentation](https://mlsanity.com/paper/2104.05970/) (2021)
- [Video Instance Segmentation with a Propose-Reduce Paradigm](https://mlsanity.com/paper/2103.13746/) (2021)
- [SOLQ: Segmenting Objects by Learning Queries](https://mlsanity.com/paper/2106.02351/) (2021)
- [Mask Transfiner for High-Quality Instance Segmentation](https://mlsanity.com/paper/2111.13673/) (2021)
- [ISTR: End-to-End Instance Segmentation with Transformers](https://mlsanity.com/paper/2105.00637/) (2021)
- [Look Closer to Segment Better: Boundary Patch Refinement for Instance Segmentation](https://mlsanity.com/paper/2104.05239/) (2021)
- [SOTR: Segmenting Objects with Transformers](https://mlsanity.com/paper/2108.06747/) (2021)
- [Instance Neural Radiance Field](https://mlsanity.com/paper/2304.04395/) (2023)
- [Learning to Detect Every Thing in an Open World](https://mlsanity.com/paper/2112.01698/) (2021)
- [AISFormer: Amodal Instance Segmentation with Transformer](https://mlsanity.com/paper/2210.06323/) (2022)
- [SOLO: A Simple Framework for Instance Segmentation](https://mlsanity.com/paper/2106.15947/) (2021)
- [FastInst: A Simple Query-Based Model for Real-Time Instance Segmentation](https://mlsanity.com/paper/2303.08594/) (2023)
- [CLUSTSEG: Clustering for Universal Segmentation](https://mlsanity.com/paper/2305.02187/) (2023)
- [Differentiable Multi-Granularity Human Representation Learning for Instance-Aware Human Semantic Parsing](https://mlsanity.com/paper/2103.04570/) (2021)
- [Fully Convolutional Networks for Panoptic Segmentation with Point-based Supervision](https://mlsanity.com/paper/2108.07682/) (2021)
- [Betrayed by Captions: Joint Caption Grounding and Generation for Open Vocabulary Instance Segmentation](https://mlsanity.com/paper/2301.00805/) (2023)
- [PolarMask++: Enhanced Polar Representation for Single-Shot Instance Segmentation and Beyond](https://mlsanity.com/paper/2105.02184/) (2021)
- [E2EC: An End-to-End Contour-based Method for High-Quality High-Speed Instance Segmentation](https://mlsanity.com/paper/2203.04074/) (2022)
- [Human Instance Matting via Mutual Guidance and Multi-Instance Refinement](https://mlsanity.com/paper/2205.10767/) (2022)
- [Multi-Scale Aligned Distillation for Low-Resolution Detection](https://mlsanity.com/paper/2109.06875/) (2021)
- [Rank & Sort Loss for Object Detection and Instance Segmentation](https://mlsanity.com/paper/2107.11669/) (2021)
- [Instance Shadow Detection with A Single-Stage Detector](https://mlsanity.com/paper/2207.04614/) (2022)
- [Instance and Panoptic Segmentation Using Conditional Convolutions](https://mlsanity.com/paper/2102.03026/) (2021)
- [DFormer: Diffusion-guided Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2306.03437/) (2023)
- [Learning Equivariant Segmentation with Instance-Unique Querying](https://mlsanity.com/paper/2210.00911/) (2022)
- [Camouflaged Instance Segmentation In-The-Wild: Dataset, Method, and Benchmark Suite](https://mlsanity.com/paper/2103.17123/) (2021)

## Abstract (arXiv)

Sliding-window object detectors that generate bounding-box object predictions over a dense, regular grid have advanced rapidly and proven popular. In contrast, modern instance segmentation approaches are dominated by methods that first detect object bounding boxes, and then crop and segment these regions, as popularized by Mask R-CNN. In this work, we investigate the paradigm of dense sliding-window instance segmentation, which is surprisingly under-explored. Our core observation is that this task is fundamentally different than other dense prediction tasks such as semantic segmentation or bounding-box object detection, as the output at every spatial location is itself a geometric structure with its own spatial dimensions. To formalize this, we treat dense instance segmentation as a prediction task over 4D tensors and present a general framework called TensorMask that explicitly captures this geometry and enables novel operators on 4D tensors. We demonstrate that the tensor view leads to large gains over baselines that ignore this structure, and leads to results comparable to Mask R-CNN. These promising results suggest that TensorMask can serve as a foundation for novel advances in dense mask prediction and a more complete understanding of the task. Code will be made available.