# Learning to Fuse Things and Stuff

arXiv: https://arxiv.org/abs/1812.01192 · HTML: https://mlsanity.com/paper/1812.01192/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Dec 2018.

## Built on by (most cited first, 26 of 26)

- [Kimera: from SLAM to Spatial Perception with 3D Dynamic Scene Graphs](https://mlsanity.com/paper/2101.06894/) (2021)
- [Video K-Net: A Simple, Strong, and Unified Baseline for Video Segmentation](https://mlsanity.com/paper/2204.04656/) (2022)
- [CMT-DeepLab: Clustering Mask Transformers for Panoptic Segmentation](https://mlsanity.com/paper/2206.08948/) (2022)
- [Bird's-Eye-View Panoptic Segmentation Using Monocular Frontal View Images](https://mlsanity.com/paper/2108.03227/) (2021)
- [Panoptic-PartFormer: Learning a Unified Model for Panoptic Part Segmentation](https://mlsanity.com/paper/2204.04655/) (2022)
- [PolyphonicFormer: Unified Query Learning for Depth-aware Video Panoptic Segmentation](https://mlsanity.com/paper/2112.02582/) (2021)
- [Fully Convolutional Networks for Panoptic Segmentation with Point-based Supervision](https://mlsanity.com/paper/2108.07682/) (2021)
- [MGNet: Monocular Geometric Scene Understanding for Autonomous Driving](https://mlsanity.com/paper/2206.13199/) (2022)
- [PanopticPartFormer++: A Unified and Decoupled View for Panoptic Part Segmentation](https://mlsanity.com/paper/2301.00954/) (2023)
- [Interactive Image Synthesis with Panoptic Layout Generation](https://mlsanity.com/paper/2203.02104/) (2022)
- [PanopticDepth: A Unified Framework for Depth-aware Panoptic Segmentation](https://mlsanity.com/paper/2206.00468/) (2022)
- [Combinatorial Optimization for Panoptic Segmentation: A Fully Differentiable Approach](https://mlsanity.com/paper/2106.03188/) (2021)
- [Visual Recognition by Request](https://mlsanity.com/paper/2207.14227/) (2022)
- [Milestones in Autonomous Driving and Intelligent Vehicles Part II: Perception and Planning](https://mlsanity.com/paper/2306.01980/) (2023)
- [Panoptic Segmentation Forecasting](https://mlsanity.com/paper/2104.03962/) (2021)
- [Multi-task Fusion for Efficient Panoptic-Part Segmentation](https://mlsanity.com/paper/2212.07671/) (2022)
- [Flying Guide Dog: Walkable Path Discovery for the Visually Impaired Utilizing Drones and Transformer-based Semantic Segmentation](https://mlsanity.com/paper/2108.07007/) (2021)
- [Hierarchical Lov'asz Embeddings for Proposal-free Panoptic Segmentation](https://mlsanity.com/paper/2106.04555/) (2021)
- [MC-PanDA: Mask Confidence for Panoptic Domain Adaptation](https://mlsanity.com/paper/2407.14110/) (2024)
- [Intra-Batch Supervision for Panoptic Segmentation on High-Resolution Images](https://mlsanity.com/paper/2304.08222/) (2023)
- [SE-PSNet: Silhouette-based Enhancement Feature for Panoptic Segmentation Network](https://mlsanity.com/paper/2107.05093/) (2021)
- [JPPF: Multi-task Fusion for Consistent Panoptic-Part Segmentation](https://mlsanity.com/paper/2311.18618/) (2023)
- [Resolving Inconsistent Semantics in Multi-Dataset Image Segmentation](https://mlsanity.com/paper/2409.09893/) (2024)
- [HyCTAS: Multi-Objective Hybrid Convolution-Transformer Architecture Search for Real-Time Image Segmentation](https://mlsanity.com/paper/2403.10413/) (2024)
- [MambaPanoptic: A Vision Mamba-based Structured State Space Framework for Panoptic Segmentation](https://mlsanity.com/paper/2605.12640/) (2026)
- [MC-PanDA++: Simpler, Stronger, and More Robust Domain-Adaptive Panoptic Segmentation](https://mlsanity.com/paper/2609.39681/) (2026)

## Abstract (arXiv)

We propose an end-to-end learning approach for panoptic segmentation, a novel task unifying instance (things) and semantic (stuff) segmentation. Our model, TASCNet, uses feature maps from a shared backbone network to predict in a single feed-forward pass both things and stuff segmentations. We explicitly constrain these two output distributions through a global things and stuff binary mask to enforce cross-task consistency. Our proposed unified network is competitive with the state of the art on several benchmarks for panoptic segmentation as well as on the individual semantic and instance segmentation tasks.