mlsanityData to 14 Oct 2026

Spatially Grounded Concept-Based Image Classification

arXiv 2510.04180 ↗First posted 5 Oct 2025cs.CV cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Deep neural networks can achieve high accuracy while relying on evidence that is hard to inspect or misaligned with the intended task. Concept Bottleneck Models (CBMs) expose human-interpretable concepts, but most treat concepts as global attributes and do not show how localized evidence is aggregated into a decision. We propose SEG-MIL-CBM, a spatially grounded CBM that decomposes each image into concept-guided regions and classifies it by attention-based aggregation of segment-level concept evidence. The same segment evidence terms form the prediction and the explanation, exposing which regions and concepts support the predicted logit without a separate post-hoc attribution module. Among evaluated CBM-family baselines, SEG-MIL-CBM improves Waterbirds worst-group accuracy from 65.1% to 72.0%, reaches 87.4% worst-group accuracy on Pawrious, remains competitive on standard recognition, and attains the best CBM accuracy on CIFAR-100 (85.3%). Segment-level faithfulness experiments on CUB further show that its learned segment ranking matches or improves over evaluated segment-ranking controls.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 26 of its 35 references that are in the map. First posted on arXiv 5 Oct 2025.

Builds on

26 of 35 references.

References · 26

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2023Segment Anythingcited by 8,234
  3. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  4. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,877
  5. 2019Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalizationcited by 1,247
  6. 2020Concept Bottleneck Modelscited by 1,032
  7. 2021Just Train Twice: Improving Group Robustness without Training Group Informationcited by 523
  8. 2022Last Layer Re-Training is Sufficient for Robustness to Spurious Correlationscited by 381
  9. 2022Visual Classification via Description from Large Language Modelscited by 336
  10. 2022Language in a Bottle: Language Model Guided Concept Bottlenecks for Interpretable Image Classificationcited by 312
  11. 2020Environment Inference for Invariant Learningcited by 306
  12. 2022Post-hoc Concept Bottleneck Modelscited by 293
14 more
  1. 2023Label-Free Concept Bottleneck Modelscited by 285
  2. 2022Correct-N-Contrast: A Contrastive Approach for Improving Robustness to Spurious Correlationscited by 172
  3. 2024Discover-then-Name: Task-Agnostic Concept Bottlenecks via Automated Concept Discoverycited by 91
  4. 2023Discover and Cure: Concept-aware Mitigation of Spurious Correlationcited by 71
  5. 2023Simple and Fast Group Robustness by Automatic Feature Reweightingcited by 68
  6. 2024VLG-CBM: Training Concept Bottleneck Models with Vision-Language Guidancecited by 65
  7. 2023Spawrious: A Benchmark for Fine Control of Spurious Correlation Biasescited by 41
  8. 2023Sparse Linear Concept Discovery Modelscited by 22
  9. 2024Decompose-and-Compose: A Compositional Approach to Mitigating Spurious Correlationcited by 18
  10. 2023Right for the Wrong Reason: Can Interpretable ML Techniques Detect Spurious Correlations?cited by 16
  11. 2024DCBM: Data-Efficient Visual Concept Bottleneck Modelscited by 16
  12. 2025Show and Tell: Visually Explainable Deep Neural Nets via Spatially-Aware Concept Bottleneck Modelscited by 12
  13. 2024Adaptive Concept Bottleneck for Foundation Models Under Distribution Shiftscited by 9
  14. 2024The Group Robustness is in the Details: Revisiting Finetuning under Spurious Correlationscited by 6

Built on by

Not cited yet.