mlsanityData to 14 Oct 2026

MetaCoCo: A New Few-Shot Classification Benchmark with Spurious Correlation

arXiv 2404.19644 ↗First posted 30 Apr 2024cs.CVcited by 5

Not on the map. Why?

Abstract (arXiv)

Out-of-distribution (OOD) problems in few-shot classification (FSC) occur when novel classes sampled from testing distributions differ from base classes drawn from training distributions, which considerably degrades the performance of deep learning models deployed in real-world applications. Recent studies suggest that the OOD problems in FSC mainly including: (a) cross-domain few-shot classification (CD-FSC) and (b) spurious-correlation few-shot classification (SC-FSC). Specifically, CD-FSC occurs when a classifier learns transferring knowledge from base classes drawn from seen training distributions but recognizes novel classes sampled from unseen testing distributions. In contrast, SC-FSC arises when a classifier relies on non-causal features (or contexts) that happen to be correlated with the labels (or concepts) in base classes but such relationships no longer hold during the model deployment. Despite CD-FSC has been extensively studied, SC-FSC remains understudied due to lack of the corresponding evaluation benchmarks. To this end, we present Meta Concept Context (MetaCoCo), a benchmark with spurious-correlation shifts collected from real-world scenarios. Moreover, to quantify the extent of spurious-correlation shifts of the presented MetaCoCo, we further propose a metric by using CLIP as a pre-trained vision-language model. Extensive experiments on the proposed benchmark are performed to evaluate the state-of-the-art methods in FSC, cross-domain shifts, and self-supervised learning. The experimental results show that the performance of the existing methods degrades significantly in the presence of spurious-correlation shifts. We open-source all codes of our benchmark and hope that the proposed MetaCoCo can facilitate future research on spurious-correlation shifts problems in FSC. The code is available at: https://github.com/remiMZ/MetaCoCo-ICLR24.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 73 of its 104 references that are in the map. First posted on arXiv 30 Apr 2024.

Builds on

73 of 104 references.

References · 73

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  4. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  5. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  6. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  7. 2017Prototypical Networks for Few-shot Learningcited by 3,015
  8. 2016Matching Networks for One Shot Learningcited by 2,255
  9. 2019Invariant Risk Minimizationcited by 1,854
  10. 2017EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classificationcited by 1,496
  11. 2018Moment Matching for Multi-Source Domain Adaptationcited by 1,331
  12. 2019Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalizationcited by 1,247
38 more
  1. 2017Learning to Compare: Relation Network for Few-Shot Learningcited by 1,143
  2. 2017ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseasescited by 972
  3. 2020Out-of-Distribution Generalization via Risk Extrapolation (REx)cited by 742
  4. 2019A Closer Look at Few-shot Classificationcited by 684
  5. 2019Skin Lesion Analysis Toward Melanoma Detection 2018: A Challenge Hosted by the International Skin Imaging Collaboration (ISIC)cited by 669
  6. 2015Causal inference using invariant prediction: identification and confidence intervalscited by 536
  7. 2018Meta-Learning with Latent Embedding Optimizationcited by 476
  8. 2020Rethinking Few-Shot Image Classification: a Good Embedding Is All You Need?cited by 437
  9. 2021Towards Out-Of-Distribution Generalization: A Surveycited by 436
  10. 2018Meta-learning with differentiable closed-form solverscited by 397
  11. 2019Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAMLcited by 359
  12. 2017Few-Shot Learning with Graph Neural Networkscited by 324
  13. 2018Meta-Transfer Learning for Few-Shot Learningcited by 306
  14. 2020Prior Guided Feature Enrichment Network for Few-Shot Segmentationcited by 271
  15. 2020The Risks of Invariant Risk Minimizationcited by 262
  16. 2016Using Deep Learning for Image-Based Plant Disease Detectioncited by 219
  17. 2019Cross Attention Network for Few-shot Classificationcited by 219
  18. 2018Learning to Propagate Labels: Transductive Propagation Network for Few-shot Learningcited by 211
  19. 2022Improving Out-of-Distribution Robustness via Selective Augmentationcited by 202
  20. 2020Cross-Domain Few-Shot Classification via Learned Feature-Wise Transformationcited by 185
  21. 2020Meta-Baseline: Exploring Simple Meta-Learning for Few-Shot Learningcited by 179
  22. 2020CrossTransformers: spatially-aware few-shot transfercited by 174
  23. 2022A Comprehensive Survey of Few-shot Learning: Evolution, Applications, Challenges, and Opportunitiescited by 163
  24. 2019Boosting Few-Shot Visual Learning with Self-Supervisioncited by 140
  25. 2020Invariant Rationalizationcited by 131
  26. 2020Negative Margin Matters: Understanding Margin in Few-shot Classificationcited by 129
  27. 2019Revisiting Local Descriptor based Image-to-Class Measure for Few-shot Learningcited by 121
  28. 2020Few-Shot Classification with Feature Map Reconstruction Networkscited by 116
  29. 2021Does Invariant Risk Minimization Capture Invariance?cited by 116
  30. 2020Interventional Few-Shot Learningcited by 112
  31. 2017Few-Shot Adversarial Domain Adaptationcited by 111
  32. 2022MetaShift: A Dataset of Datasets for Evaluating Contextual Distribution Shifts and Training Conflictscited by 81
  33. 2021Cross-domain Few-shot Learning with Task-specific Adapterscited by 80
  34. 2019When Does Self-supervision Improve Few-shot Learning?cited by 78
  35. 2021Relational Embedding for Few-Shot Classificationcited by 74
  36. 2021Cross-Domain Few-Shot Classification via Adversarial Task Augmentationcited by 63
  37. 2021Learning a Universal Template for Few-shot Dataset Generalizationcited by 61
  38. 2020Explanation-Guided Training for Cross-Domain Few-Shot Classificationcited by 58

50 most cited of 73 All 73 (JSON)

Built on by

Cited by year

Citing papers per year: 2024: 3; 2025: 2.

5 papers.

  1. 2024Navigating Shortcuts, Spurious Correlations, and Confounders: From Origins via Detection to Mitigationcited by 25
  2. 2025Safety Mirage: How Spurious Correlations Undermine VLM Safety Fine-Tuning and Can Be Mitigated by Machine Unlearningcited by 16
  3. 2024Benchmarking Spurious Bias in Few-Shot Image Classifierscited by 6
  4. 2024Cross-Modal Few-Shot Learning: a Generative Transfer Learning Frameworkcited by 2
  5. 2025Spurious-Aware Prototype Refinement for Reliable Out-of-Distribution Detectioncited by 2

Similar, not cited

Close in wording, no citation either way

  1. 2025Enabling Validation for Robust Few-Shot Recognition
  2. 2024Making Large Vision Language Models to be Good Few-shot Learners
  3. 2025Rethinking Few Shot CLIP Benchmarks: A Critical Analysis in the Inductive Setting
  4. 2025Solving Semi-Supervised Few-Shot Learning from an Auto-Annotation Perspective
  5. 2023Cross-Level Distillation and Feature Denoising for Cross-Domain Few-Shot Classification
  6. 2024Few-Shot Recognition via Stage-Wise Retrieval-Augmented Finetuning
  7. 2022FS-BAN: Born-Again Networks for Domain Generalization Few-Shot Classification
  8. 2023A framework for benchmarking class-out-of-distribution detection and its application to ImageNet