mlsanityData to 7 Oct 2026

Trapped in texture bias? A large scale comparison of deep instance segmentation

arXiv 2401.09109 ↗First posted 17 Jan 2024cs.CVcited by 10

Not on the map. Why?

Abstract (arXiv)

Do deep learning models for instance segmentation generalize to novel objects in a systematic way? For classification, such behavior has been questioned. In this study, we aim to understand if certain design decisions such as framework, architecture or pre-training contribute to the semantic understanding of instance segmentation. To answer this question, we consider a special case of robustness and compare pre-trained models on a challenging benchmark for object-centric, out-of-distribution texture. We do not introduce another method in this work. Instead, we take a step back and evaluate a broad range of existing literature. This includes Cascade and Mask R-CNN, Swin Transformer, BMask, YOLACT(++), DETR, BCNet, SOTR and SOLOv2. We find that YOLACT++, SOTR and SOLOv2 are significantly more robust to out-of-distribution texture than other frameworks. In addition, we show that deeper and dynamic architectures improve robustness whereas training schedules, data augmentation and pre-training have only a minor impact. In summary we evaluate 68 models on 61 versions of MS COCO for a total of 4148 evaluations.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 69 of its 77 references that are in the map. First posted on arXiv 17 Jan 2024.

Builds on

69 of 77 references.

References · 69

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  5. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  6. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  7. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  8. 2020End-to-End Object Detection with Transformerscited by 7,384
  9. 2017Mask R-CNNcited by 7,259
  10. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  11. 2016Feature Pyramid Networks for Object Detectioncited by 5,021
  12. 2017Squeeze-and-Excitation Networkscited by 4,763
38 more
  1. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,857
  2. 2018YOLOv3: An Incremental Improvementcited by 2,652
  3. 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,540
  4. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,282
  5. 2017Arbitrary Style Transfer in Real-time with Adaptive Instance Normalizationcited by 1,998
  6. 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,838
  7. 2020YOLOv4: Optimal Speed and Accuracy of Object Detectioncited by 1,624
  8. 2019FCOS: Fully Convolutional One-Stage Object Detectioncited by 1,573
  9. 2019Do ImageNet Classifiers Generalize to ImageNet?cited by 1,507
  10. 2017Deformable Convolutional Networkscited by 1,481
  11. 2018ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustnesscited by 1,430
  12. 2019Natural Adversarial Examplescited by 1,401
  13. 2017Cascade R-CNN: Delving into High Quality Object Detectioncited by 1,275
  14. 2020Designing Network Design Spacescited by 954
  15. 2020What Makes for Good Views for Contrastive Learning?cited by 843
  16. 2019Self-Supervised Learning of Pretext-Invariant Representationscited by 806
  17. 2018Path Aggregation Network for Instance Segmentationcited by 796
  18. 2014Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognitioncited by 719
  19. 2018CCNet: Criss-Cross Attention for Semantic Segmentationcited by 717
  20. 2018Deformable ConvNets v2: More Deformable, Better Resultscited by 666
  21. 2020Simple Copy-Paste is a Strong Data Augmentation Method for Instance Segmentationcited by 595
  22. 2019YOLACT: Real-time Instance Segmentationcited by 441
  23. 2019PointRend: Image Segmentation as Renderingcited by 439
  24. 2019CSPNet: A New Backbone that can Enhance Learning Capability of CNNcited by 381
  25. 2019Benchmarking Robustness in Object Detection: Autonomous Driving when Winter is Comingcited by 332
  26. 2020Conditional Convolutions for Instance Segmentationcited by 314
  27. 2019SOLO: Segmenting Objects by Locationscited by 289
  28. 2019Mask Scoring R-CNNcited by 265
  29. 2019Approximating CNNs with Bag-of-local-Features models works surprisingly well on ImageNetcited by 260
  30. 2018Generalisation in humans and deep neural networkscited by 237
  31. 2020On the Binding Problem in Artificial Neural Networkscited by 230
  32. 2020Scaled-YOLOv4: Scaling Cross Stage Partial Networkcited by 211
  33. 2021Partial success in closing the gap between human and machine visioncited by 185
  34. 2021On the Robustness of Vision Transformers to Adversarial Examplescited by 167
  35. 2020Seesaw Loss for Long-Tailed Instance Segmentationcited by 146
  36. 2019Benchmarking the Robustness of Semantic Segmentation Modelscited by 111
  37. 2021Are Convolutional Neural Networks or Transformers more like human vision?cited by 109
  38. 2020The Devil is in Classification: A Simple Framework for Long-tail Object Detection and Instance Segmentationcited by 108

50 most cited of 69 All 69 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 5; 2026: 2.

10 papers.

  1. 2024Benchmarking Segmentation Models with Mask-Preserved Attribute Editingcited by 6
  2. 2025Shape Bias and Robustness Evaluation via Cue Decomposition for Image Classification and Segmentationcited by 3
  3. 2025Decomposing and Revising What Language Models Generatecited by 2
  4. 2026Benchmarking Semantic Segmentation Models via Appearance and Geometry Attribute Editingcited by 2
  5. 2025On Background Bias of Post-Hoc Concept Embeddings in Computer Vision DNNscited by 1
  6. 2025Transferring Styles for Reduced Texture Bias and Improved Robustness in Semantic Segmentation Networkscited by 1
  7. 2024On the Influence of Shape, Texture and Color for Learning Semantic Segmentationcited by 0
  8. 2024Contrast All the Time: Learning Time Series Representation from Temporal Consistencycited by 0
  9. 2025LSP-ST: Ladder Shape-Biased Side-Tuning for Robust Infrared Small Target Detectioncited by 0
  10. 2026Differences in Detection: Explainability Where it Matterscited by 0