mlsanityData to 7 Oct 2026

On the Influence of Shape, Texture and Color for Learning Semantic Segmentation

arXiv 2410.14878 ↗First posted 18 Oct 2024cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Recent research has investigated the shape and texture biases of pre-trained deep neural networks (DNNs) in image classification. Those works test how much a trained DNN relies on specific image cues like texture. The present study shifts the focus to understanding the cue influence during training, analyzing what DNNs can learn from shape, texture, and color cues in absence of the others; investigating their individual and combined influence on the learning success. We analyze these cue influences at multiple levels by decomposing datasets into cue-specific versions. Addressing semantic segmentation, we learn the given task from these reduced cue datasets, creating cue experts. Early fusion of cues is performed by constructing appropriate datasets. This is complemented by a late fusion of experts which allows us to study cue influence location-dependent on pixel level. Experiments on Cityscapes, PASCAL Context, and a synthetic CARLA dataset show that while no single cue dominates, the shape + color expert predominantly improves the prediction of small objects and border pixels. The cue performance order is consistent for the tested convolutional and transformer architecture, indicating similar cue extraction capabilities, although pre-trained transformers are said to be more biased towards shape than convolutional neural networks.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 81 of its 146 references that are in the map. First posted on arXiv 18 Oct 2024.

Builds on

81 of 146 references.

References · 81

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  4. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  5. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  6. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  7. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  8. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  9. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  10. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
  11. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  12. 2018Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentationcited by 3,210
38 more
  1. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,923
  2. 2020Exploring Simple Siamese Representation Learningcited by 2,853
  3. 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
  4. 2017Rethinking Atrous Convolution for Semantic Image Segmentationcited by 2,154
  5. 2020Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecastingcited by 2,141
  6. 2017Arbitrary Style Transfer in Real-time with Adaptive Instance Normalizationcited by 1,998
  7. 2022A Time Series is Worth 64 Words: Long-term Forecasting with Transformerscited by 1,981
  8. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,591
  9. 2018ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustnesscited by 1,430
  10. 2022Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questionscited by 911
  11. 2021MuSiQue: Multihop Questions via Single-hop Question Compositioncited by 893
  12. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 736
  13. 2015Holistically-Nested Edge Detectioncited by 590
  14. 2023Enabling Large Language Models to Generate Text with Citationscited by 584
  15. 2021TS2Vec: Towards Universal Representation of Time Seriescited by 463
  16. 2020A Transformer-based Framework for Multivariate Time Series Representation Learningcited by 449
  17. 2021Intriguing Properties of Vision Transformerscited by 446
  18. 2018The UCR Time Series Archivecited by 421
  19. 2018Detecting Spacecraft Anomalies Using LSTMs and Nonparametric Dynamic Thresholdingcited by 407
  20. 2021Time-Series Representation Learning via Temporal and Contextual Contrastingcited by 342
  21. 2021With a Little Help from My Friends: Nearest-Neighbor Contrastive Learning of Visual Representationscited by 324
  22. 2022RARR: Researching and Revising What Language Models Say, Using Language Modelscited by 286
  23. 2023Can We Edit Factual Knowledge by In-Context Learning?cited by 284
  24. 2019Unsupervised Scalable Representation Learning for Multivariate Time Seriescited by 267
  25. 2019Approximating CNNs with Bag-of-local-Features models works surprisingly well on ImageNetcited by 260
  26. 2021Measuring Attribution in Natural Language Generation Modelscited by 253
  27. 2022Self-Supervised Contrastive Pre-Training For Time Series via Time-Frequency Consistencycited by 246
  28. 2021Unsupervised Representation Learning for Time Series with Temporal Neighborhood Codingcited by 223
  29. 2020Representation Learning via Invariant Causal Mechanismscited by 197
  30. 2021Partial success in closing the gap between human and machine visioncited by 185
  31. 2023Large Language Models are Versatile Decomposers: Decompose Evidence and Questions for Table-based Reasoningcited by 177
  32. 2016Exploring the Limits of Language Modelingcited by 172
  33. 2020CLOCS: Contrastive Learning of Cardiac Signals Across Space, Time, and Patientscited by 170
  34. 2024Fine-grained Hallucination Detection and Editing for Language Modelscited by 146
  35. 2022CoST: Contrastive Learning of Disentangled Seasonal-Trend Representations for Time Series Forecastingcited by 143
  36. 2021Decoupled Contrastive Learningcited by 141
  37. 2023SimMTM: A Simple Pre-Training Framework for Masked Time-Series Modelingcited by 112
  38. 2023Complex Claim Verification with Evidence Retrieved in the Wildcited by 110

50 most cited of 81 All 81 (JSON)

Built on by

Not cited yet.