# Learning to Segment via Cut-and-Paste

arXiv: https://arxiv.org/abs/1803.06414 · HTML: https://mlsanity.com/paper/1803.06414/

## Key facts

- Cited by 42 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Mar 2018.

## Built on by (most cited first, 30 of 42)

- [Pre-training without Natural Images](https://mlsanity.com/paper/2101.08515/) (2021)
- [Context Decoupling Augmentation for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2103.01795/) (2021)
- [Making Images Real Again: A Comprehensive Survey on Deep Image Composition](https://mlsanity.com/paper/2106.14490/) (2021)
- [DSP: Dual Soft-Paste for Unsupervised Domain Adaptive Semantic Segmentation](https://mlsanity.com/paper/2107.09600/) (2021)
- [DaTaSeg: Taming a Universal Multi-Dataset Multi-Task Segmentation Model](https://mlsanity.com/paper/2306.01736/) (2023)
- [OPA: Object Placement Assessment Dataset](https://mlsanity.com/paper/2107.01889/) (2021)
- [Shatter and Gather: Learning Referring Image Segmentation with Text Supervision](https://mlsanity.com/paper/2308.15512/) (2023)
- [Decoder Denoising Pretraining for Semantic Segmentation](https://mlsanity.com/paper/2205.11423/) (2022)
- [MOVE: Unsupervised Movable Object Segmentation and Detection](https://mlsanity.com/paper/2210.07920/) (2022)
- [Thinking Outside the BBox: Unconstrained Generative Object Compositing](https://mlsanity.com/paper/2409.04559/) (2024)
- [Weakly supervised segmentation with cross-modality equivariant constraints](https://mlsanity.com/paper/2104.02488/) (2021)
- [Learning Interactive Driving Policies via Data-driven Simulation](https://mlsanity.com/paper/2111.12137/) (2021)
- [Unsupervised Object Localization in the Era of Self-Supervised ViTs: A Survey](https://mlsanity.com/paper/2310.12904/) (2023)
- [MMNet: Multi-Collaboration and Multi-Supervision Network for Sequential Deepfake Detection](https://mlsanity.com/paper/2307.02733/) (2023)
- [Scaling Backwards: Minimal Synthetic Pre-training?](https://mlsanity.com/paper/2408.00677/) (2024)
- [The surprising impact of mask-head architecture on novel class segmentation](https://mlsanity.com/paper/2104.00613/) (2021)
- [Fast Object Placement Assessment](https://mlsanity.com/paper/2205.14280/) (2022)
- [Amodal Cityscapes: A New Dataset, its Generation, and an Amodal Semantic Segmentation Challenge Baseline](https://mlsanity.com/paper/2206.00527/) (2022)
- [Exploring Localization for Self-supervised Fine-grained Contrastive Learning](https://mlsanity.com/paper/2106.15788/) (2021)
- [Characterizing and Improving the Robustness of Self-Supervised Learning through Background Augmentations](https://mlsanity.com/paper/2103.12719/) (2021)
- [Learning with Free Object Segments for Long-Tailed Instance Segmentation](https://mlsanity.com/paper/2202.11124/) (2022)
- [Learning Foreground-Background Segmentation from Improved Layered GANs](https://mlsanity.com/paper/2104.00483/) (2021)
- [Reducing Annotating Load: Active Learning with Synthetic Images in Surgical Instrument Segmentation](https://mlsanity.com/paper/2108.03534/) (2021)
- [Towards Automated Polyp Segmentation Using Weakly- and Semi-Supervised Learning and Deformable Transformers](https://mlsanity.com/paper/2211.11847/) (2022)
- [Improving Crowded Object Detection via Copy-Paste](https://mlsanity.com/paper/2211.12110/) (2022)
- [Foreground-Background Separation through Concept Distillation from Generative Image Foundation Models](https://mlsanity.com/paper/2212.14306/) (2022)
- [Hard-aware Instance Adaptive Self-training for Unsupervised Cross-domain Semantic Segmentation](https://mlsanity.com/paper/2302.06992/) (2023)
- [Edge-competing Pathological Liver Vessel Segmentation with Limited Labels](https://mlsanity.com/paper/2108.00384/) (2021)
- [Cut-Paste Consistency Learning for Semi-Supervised Lesion Segmentation](https://mlsanity.com/paper/2210.00191/) (2022)
- [DiffPop: Plausibility-Guided Object Placement Diffusion for Image Composition](https://mlsanity.com/paper/2406.07852/) (2024)

## Abstract (arXiv)

This paper presents a weakly-supervised approach to object instance segmentation. Starting with known or predicted object bounding boxes, we learn object masks by playing a game of cut-and-paste in an adversarial learning setup. A mask generator takes a detection box and Faster R-CNN features, and constructs a segmentation mask that is used to cut-and-paste the object into a new image location. The discriminator tries to distinguish between real objects, and those cut and pasted via the generator, giving a learning signal that leads to improved object masks. We verify our method experimentally using Cityscapes, COCO, and aerial image datasets, learning to segment objects without ever having seen a mask in training. Our method exceeds the performance of existing weakly supervised methods, without requiring hand-tuned segment proposals, and reaches 90% of supervised performance.