# On the Importance of Visual Context for Data Augmentation in Scene Understanding

arXiv: https://arxiv.org/abs/1809.02492 · HTML: https://mlsanity.com/paper/1809.02492/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Sep 2018.

## Built on by (most cited first, 19 of 19)

- [Making Images Real Again: A Comprehensive Survey on Deep Image Composition](https://mlsanity.com/paper/2106.14490/) (2021)
- [Hyperbolic Contrastive Learning for Visual Representations beyond Objects](https://mlsanity.com/paper/2212.00653/) (2022)
- [Improving Semi-Supervised and Domain-Adaptive Semantic Segmentation with Self-Supervised Depth Estimation](https://mlsanity.com/paper/2108.12545/) (2021)
- [Thinking Outside the BBox: Unconstrained Generative Object Compositing](https://mlsanity.com/paper/2409.04559/) (2024)
- [Self-supervised Transparent Liquid Segmentation for Robotic Pouring](https://mlsanity.com/paper/2203.01538/) (2022)
- [Sill-Net: Feature Augmentation with Separated Illumination Representation](https://mlsanity.com/paper/2102.03539/) (2021)
- [Contextualizing Meta-Learning via Learning to Decompose](https://mlsanity.com/paper/2106.08112/) (2021)
- [Traffic Context Aware Data Augmentation for Rare Object Detection in Autonomous Driving](https://mlsanity.com/paper/2205.00376/) (2022)
- [AnimateScene: Camera-controllable Animation in Any Scene](https://mlsanity.com/paper/2508.05982/) (2025)
- [Data augmentation by morphological mixup for solving Raven's Progressive Matrices](https://mlsanity.com/paper/2103.05222/) (2021)
- [Randomize to Generalize: Domain Randomization for Runway FOD Detection](https://mlsanity.com/paper/2309.13264/) (2023)
- [ContextMix: A context-aware data augmentation method for industrial visual inspection systems](https://mlsanity.com/paper/2401.10050/) (2024)
- [Evaluating the Efficacy of Cut-and-Paste Data Augmentation in Semantic Segmentation for Satellite Imagery](https://mlsanity.com/paper/2404.05693/) (2024)
- [Scene-Aware Location Modeling for Data Augmentation in Automotive Object Detection](https://mlsanity.com/paper/2504.17076/) (2025)
- [3rd Place Scheme on Instance Segmentation Track of ICCV 2021 VIPriors Challenges](https://mlsanity.com/paper/2110.00242/) (2021)
- [SemAug: Semantically Meaningful Image Augmentations for Object Detection Through Language Grounding](https://mlsanity.com/paper/2208.07407/) (2022)
- [DepGAN: Leveraging Depth Maps for Handling Occlusions and Transparency in Image Composition](https://mlsanity.com/paper/2407.11890/) (2024)
- [A Survey on Cell Nuclei Instance Segmentation and Classification: Leveraging Context and Attention](https://mlsanity.com/paper/2407.18673/) (2024)
- [presto: Efficient, Training-free, and Open-world Object Placement via Imaginary Search](https://mlsanity.com/paper/2608.21543/) (2026)

## Abstract (arXiv)

Performing data augmentation for learning deep neural networks is known to be important for training visual recognition systems. By artificially increasing the number of training examples, it helps reducing overfitting and improves generalization. While simple image transformations can already improve predictive performance in most vision tasks, larger gains can be obtained by leveraging task-specific prior knowledge. In this work, we consider object detection, semantic and instance segmentation and augment the training images by blending objects in existing scenes, using instance segmentation annotations. We observe that randomly pasting objects on images hurts the performance, unless the object is placed in the right context. To resolve this issue, we propose an explicit context model by using a convolutional neural network, which predicts whether an image region is suitable for placing a given object or not. In our experiments, we show that our approach is able to improve object detection, semantic and instance segmentation on the PASCAL VOC12 and COCO datasets, with significant gains in a limited annotation scenario, i.e. when only one category is annotated. We also show that the method is not limited to datasets that come with expensive pixel-wise instance annotations and can be used when only bounding boxes are available, by employing weakly-supervised learning for instance masks approximation.