# Convolutional Feature Masking for Joint Object and Stuff Segmentation

arXiv: https://arxiv.org/abs/1412.1283 · HTML: https://mlsanity.com/paper/1412.1283/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Dec 2014.

## Built on by (most cited first, 30 of 41)

- [Per-Pixel Classification is Not All You Need for Semantic Segmentation](https://mlsanity.com/paper/2107.06278/) (2021)
- [OneFormer: One Transformer to Rule Universal Image Segmentation](https://mlsanity.com/paper/2211.06220/) (2022)
- [Masked World Models for Visual Control](https://mlsanity.com/paper/2206.14244/) (2022)
- [Perturbed and Strict Mean Teachers for Semi-supervised Semantic Segmentation](https://mlsanity.com/paper/2111.12903/) (2021)
- [Learning Open-World Object Proposals without Learning to Classify](https://mlsanity.com/paper/2108.06753/) (2021)
- [Re-distributing Biased Pseudo Labels for Semi-supervised Semantic Segmentation: A Baseline Investigation](https://mlsanity.com/paper/2107.11279/) (2021)
- [GROUNDHOG: Grounding Large Language Models to Holistic Segmentation](https://mlsanity.com/paper/2402.16846/) (2024)
- [Drive Anywhere: Generalizable End-to-end Autonomous Driving with Multi-modal Foundation Models](https://mlsanity.com/paper/2310.17642/) (2023)
- [On Advantages of Mask-level Recognition for Outlier-aware Segmentation](https://mlsanity.com/paper/2301.03407/) (2023)
- [Unbiased Subclass Regularization for Semi-Supervised Semantic Segmentation](https://mlsanity.com/paper/2203.10026/) (2022)
- [From Forks to Forceps: A New Framework for Instance Segmentation of Surgical Instruments](https://mlsanity.com/paper/2211.16200/) (2022)
- [OpenPSG: Open-set Panoptic Scene Graph Generation via Large Multimodal Models](https://mlsanity.com/paper/2407.11213/) (2024)
- [Concept-Based Techniques for "Musicologist-friendly" Explanations in a Deep Music Classifier](https://mlsanity.com/paper/2208.12485/) (2022)
- [HAPNet: Toward Superior RGB-Thermal Scene Parsing via Hybrid, Asymmetric, and Progressive Heterogeneous Feature Fusion](https://mlsanity.com/paper/2404.03527/) (2024)
- [Masking Strategies for Background Bias Removal in Computer Vision Models](https://mlsanity.com/paper/2308.12127/) (2023)
- [Probabilistic Graph Attention Network with Conditional Kernels for Pixel-Wise Prediction](https://mlsanity.com/paper/2101.02843/) (2021)
- [Occlusion-Aware Seamless Segmentation](https://mlsanity.com/paper/2407.02182/) (2024)
- [Transformer-Unet: Raw Image Processing with Unet](https://mlsanity.com/paper/2109.08417/) (2021)
- [Recognizing Scenes from Novel Viewpoints](https://mlsanity.com/paper/2112.01520/) (2021)
- [OAMixer: Object-aware Mixing Layer for Vision Transformers](https://mlsanity.com/paper/2212.06595/) (2022)
- [Distributed Semantic Segmentation with Efficient Joint Source and Task Decoding](https://mlsanity.com/paper/2407.11224/) (2024)
- [Unlocking Constraints: Source-Free Occlusion-Aware Seamless Segmentation](https://mlsanity.com/paper/2506.21198/) (2025)
- [See Less, Drive Better: Generalizable End-to-End Autonomous Driving via Foundation Models Stochastic Patch Selection](https://mlsanity.com/paper/2601.10707/) (2026)
- [Content-Aware Convolutional Neural Networks](https://mlsanity.com/paper/2106.15797/) (2021)
- [Panoptic-aware Image-to-Image Translation](https://mlsanity.com/paper/2112.01926/) (2021)
- [End-to-End Instance Edge Detection](https://mlsanity.com/paper/2204.02898/) (2022)
- [Variational Structured Attention Networks for Deep Visual Representation Learning](https://mlsanity.com/paper/2103.03510/) (2021)
- [Semantics-Depth-Symbiosis: Deeply Coupled Semi-Supervised Learning of Semantics and Depth](https://mlsanity.com/paper/2206.10562/) (2022)
- [Tunable Hybrid Proposal Networks for the Open World](https://mlsanity.com/paper/2208.11050/) (2022)
- [OASIS: Automated Assessment of Urban Pedestrian Paths at Scale](https://mlsanity.com/paper/2303.02287/) (2023)

## Abstract (arXiv)

The topic of semantic segmentation has witnessed considerable progress due to the powerful features learned by convolutional neural networks (CNNs). The current leading approaches for semantic segmentation exploit shape information by extracting CNN features from masked image regions. This strategy introduces artificial boundaries on the images and may impact the quality of the extracted features. Besides, the operations on the raw image domain require to compute thousands of networks on a single image, which is time-consuming. In this paper, we propose to exploit shape information via masking convolutional features. The proposal segments (e.g., super-pixels) are treated as masks on the convolutional feature maps. The CNN features of segments are directly masked out from these maps and used to train classifiers for recognition. We further propose a joint method to handle objects and "stuff" (e.g., grass, sky, water) in the same framework. State-of-the-art results are demonstrated on benchmarks of PASCAL VOC and new PASCAL-CONTEXT, with a compelling computational speed.