mlsanityData to 7 Oct 2026

Bringing the Context Back into Object Recognition, Robustly

arXiv 2411.15933 ↗First posted 24 Nov 2024cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

In object recognition, both the subject of interest (referred to as foreground, FG, for simplicity) and its surrounding context (background, BG) may play an important role. However, standard supervised learning often leads to unintended over-reliance on the BG, limiting model robustness in real-world deployment settings. The problem is mainly addressed by suppressing the BG, sacrificing context information for improved generalization. We propose "Localize to Recognize Robustly" (L2R2), a novel recognition approach which exploits the benefits of context-aware classification while maintaining robustness to distribution shifts. L2R2 leverages advances in zero-shot detection to localize the FG before recognition. It improves the performance of both standard recognition with supervised training, as well as multimodal zero-shot recognition with VLMs, while being robust to long-tail BGs and distribution shifts. The results confirm localization before recognition is possible for a wide range of datasets and they highlight the limits of object detection on others

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 44 of its 71 references that are in the map. First posted on arXiv 24 Nov 2024.

Builds on

44 of 71 references.

References · 44

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  4. 2023Segment Anythingcited by 8,158
  5. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  6. 2017On Calibration of Modern Neural Networkscited by 4,860
  7. 2022A ConvNet for the 2020scited by 3,545
  8. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
  9. 2024SAM 2: Segment Anything in Images and Videoscited by 2,784
  10. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,338
  11. 2020Shortcut Learning in Deep Neural Networkscited by 1,928
  12. 2019Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalizationcited by 1,232
32 more
  1. 2016Deep CORAL: Correlation Alignment for Deep Domain Adaptationcited by 1,194
  2. 2020Out-of-Distribution Generalization via Risk Extrapolation (REx)cited by 737
  3. 2023ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoderscited by 703
  4. 2023Segment Anything in High Qualitycited by 373
  5. 2020Noise or Signal: The Role of Image Backgrounds in Object Recognitioncited by 342
  6. 2023Scaling Open-Vocabulary Object Detectioncited by 335
  7. 2023Fast Segment Anythingcited by 286
  8. 2021Gradient Matching for Domain Generalizationcited by 258
  9. 2019Adversarial Domain Adaptation with Domain Mixupcited by 250
  10. 2022Improving Out-of-Distribution Robustness via Selective Augmentationcited by 202
  11. 2023Putting the Object Back into Video Object Segmentationcited by 182
  12. 2023BioCLIP: A Vision Foundation Model for the Tree of Lifecited by 159
  13. 2021Salient ImageNet: How to discover spurious features in Deep Learning?cited by 114
  14. 2022MaskTune: Mitigating Spurious Correlations by Forcing to Explorecited by 66
  15. 2022When does dough become a bagel? Analyzing the remaining mistakes on ImageNetcited by 60
  16. 2022A Comprehensive Study of Image Classification Model Sensitivity to Foregrounds, Backgrounds, and Visual Attributescited by 56
  17. 2018Not Using the Car to See the Sidewalk: Quantifying and Controlling the Effects of Context in Classification and Segmentationcited by 49
  18. 2023Spawrious: A Benchmark for Fine Control of Spurious Correlation Biasescited by 41
  19. 2016Object Recognition with and without Objectscited by 39
  20. 2024A Sober Look at the Robustness of CLIPs to Spurious Featurescited by 38
  21. 2023ImageNet-Hard: The Hardest Images Remaining from a Study of the Power of Zoom and Spatial Biases in Image Classificationcited by 33
  22. 2022Invariant Causal Mechanisms through Distribution Matchingcited by 31
  23. 2021Danish Fungi 2020 -- Not Just Another Image Recognition Datasetcited by 27
  24. 2023Fine-grained Visual Classification with High-temperature Refinement and Background Suppressioncited by 22
  25. 2022The Two Dimensions of Worst-case Training and the Integrated Effect for Out-of-domain Generalizationcited by 21
  26. 2023Masking Strategies for Background Bias Removal in Computer Vision Modelscited by 9
  27. 2024Animal Identification with Independent Foreground and Background Modelingcited by 7
  28. 2024FungiTastic: A multi-modal dataset and benchmark for image categorizationcited by 7
  29. 2022CLAD: A Contrastive Learning based Approach for Background Debiasingcited by 6
  30. 2023Mitigating the Effect of Incidental Correlations on Part-based Learningcited by 6
  31. 2023ASPIRE: Language-Guided Data Augmentation for Improving Robustness Against Spurious Correlationscited by 5
  32. 2022Detecting out-of-context objects using contextual cuescited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Robust Context-Aware Object Recognition
  2. 2026FOCUS: Forcing In-Context Object Localization through Visual Support Constraints and Policy Optimization
  3. 2026ProCal: Inference-Time Proposal Calibration for Open-Vocabulary Object Detection
  4. 2025Zero-Shot Scene Understanding for Automatic Target Recognition Using Large Vision-Language Models
  5. 2024Teaching VLMs to Localize Specific Objects from In-context Examples
  6. 2023CLIP the Gap: A Single Domain Generalization Approach for Object Detection
  7. 2025Enabling Validation for Robust Few-Shot Recognition
  8. 2026Personalized Object Identification and Localization via In-Context Inference with Vision-Language Models