# VFM4SDG: Unveiling the Power of VFMs for Single-Domain Generalized Object Detection

arXiv: https://arxiv.org/abs/2604.21502 · HTML: https://mlsanity.com/paper/2604.21502/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 45 of its 55 references that are in the map.
- First posted on arXiv 23 Apr 2026.

## Builds on (most cited first, 30 of 45)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [SAM 2: Segment Anything in Images and Videos](https://mlsanity.com/paper/2408.00714/) (2024)
- [BEiT: BERT Pre-Training of Image Transformers](https://mlsanity.com/paper/2106.08254/) (2021)
- [An Empirical Study of Training Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.02057/) (2021)
- [DINOv3](https://mlsanity.com/paper/2508.10104/) (2025)
- [DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection](https://mlsanity.com/paper/2203.03605/) (2022)
- [Grounded Language-Image Pre-training](https://mlsanity.com/paper/2112.03857/) (2021)
- [SAM 3: Segment Anything with Concepts](https://mlsanity.com/paper/2511.16719/) (2025)
- [iBOT: Image BERT Pre-Training with Online Tokenizer](https://mlsanity.com/paper/2111.07832/) (2021)
- [Vision Transformer Adapter for Dense Predictions](https://mlsanity.com/paper/2205.08534/) (2022)
- [DeiT III: Revenge of the ViT](https://mlsanity.com/paper/2204.07118/) (2022)
- [Learning to Learn Single Domain Generalization](https://mlsanity.com/paper/2003.13216/) (2020)
- [DETRs with Collaborative Hybrid Assignments Training](https://mlsanity.com/paper/2211.12860/) (2022)
- [Learning to Diversify for Single Domain Generalization](https://mlsanity.com/paper/2108.11726/) (2021)
- [Exact Feature Distribution Matching for Arbitrary Style Transfer and Domain Generalization](https://mlsanity.com/paper/2203.07740/) (2022)
- [Progressive Domain Expansion Network for Single Domain Generalization](https://mlsanity.com/paper/2103.16050/) (2021)
- [Deconstructing Denoising Diffusion Models for Self-Supervised Learning](https://mlsanity.com/paper/2401.14404/) (2024)
- [CLIP the Gap: A Single Domain Generalization Approach for Object Detection](https://mlsanity.com/paper/2301.05499/) (2023)
- [Adversarially Adaptive Normalization for Single Domain Generalization](https://mlsanity.com/paper/2106.01899/) (2021)
- [WildNet: Learning Domain Generalized Semantic Segmentation from the Wild](https://mlsanity.com/paper/2204.01446/) (2022)
- [Meta-causal Learning for Single Domain Generalization](https://mlsanity.com/paper/2304.03709/) (2023)
- [Is ImageNet worth 1 video? Learning strong image encoders from 1 long unlabelled video](https://mlsanity.com/paper/2310.08584/) (2023)
- [Attention Consistency on Visual Corruptions for Single-Source Domain Generalization](https://mlsanity.com/paper/2204.13091/) (2022)
- [Improving Single Domain-Generalized Object Detection: A Focus on Diversification and Alignment](https://mlsanity.com/paper/2405.14497/) (2024)
- [Unbiased Faster R-CNN for Single-source Domain Generalized Object Detection](https://mlsanity.com/paper/2405.15225/) (2024)

## Abstract (arXiv)

Real-world weather, illumination, and imaging variations often induce severe domain shifts, degrading single-source detectors in unseen environments. Existing single-domain generalized object detection (SDGOD) methods mainly rely on data augmentation or domain-invariant learning, while largely overlooking how domain shift disrupts detector prediction stability. Through analytical experiments, we find that performance degradation is mainly dominated by increasing missed detections. Further analysis shows that this phenomenon stems from reduced cross-domain stability in DETR-style detectors: domain shift disrupts encoder-side object-background and inter-instance relations, and further weakens the semantic-spatial binding between decoder queries and real objects. Motivated by this, we find that vision foundation models (VFMs) still preserve stable relational structures and object responses under severe shifts, making them suitable cross-domain stability priors to compensate for detector degradation. To this end, we propose VFM4SDG, a dual-prior learning framework for SDGOD, which introduces a frozen VFM into encoder representation learning and decoder query modeling. Specifically, we propose Cross-domain Stable Relational Prior Distillation to distill stable object-background and inter-instance relations from the VFM into the encoder, compensating for relational degradation. Meanwhile, we propose Semantic-Contextual Prior-based Query Enhancement, which injects category semantic prototypes and global object context into queries before they enter the decoder layer, enhancing semantic-spatial query-object binding stability. Extensive experiments show that VFM4SDG significantly outperforms existing advanced methods on standard SDGOD benchmarks and two mainstream DETR-based detection frameworks, demonstrating its effectiveness, robustness, and generality.