mlsanityData to 7 Oct 2026

Stronger, Fewer, & Superior: Harnessing Vision Foundation Models for Domain Generalized Semantic Segmentation

arXiv 2312.04265 ↗First posted 7 Dec 2023cs.CVcited by 82

Not on the map. Why?

Abstract (arXiv)

In this paper, we first assess and harness various Vision Foundation Models (VFMs) in the context of Domain Generalized Semantic Segmentation (DGSS). Driven by the motivation that Leveraging Stronger pre-trained models and Fewer trainable parameters for Superior generalizability, we introduce a robust fine-tuning approach, namely Rein, to parameter-efficiently harness VFMs for DGSS. Built upon a set of trainable tokens, each linked to distinct instances, Rein precisely refines and forwards the feature maps from each layer to the next layer within the backbone. This process produces diverse refinements for different categories within a single image. With fewer trainable parameters, Rein efficiently fine-tunes VFMs for DGSS tasks, surprisingly surpassing full parameter fine-tuning. Extensive experiments across various settings demonstrate that Rein significantly outperforms state-of-the-art methods. Remarkably, with just an extra 1% of trainable parameters within the frozen backbone, Rein achieves a mIoU of 78.4% on the Cityscapes, without accessing any real urban-scene datasets.Code is available at https://github.com/w1oves/Rein.git.

Key facts. Cited by 82 papers in the map (data through 7 Oct 2026). Builds on 57 of its 87 references that are in the map. First posted on arXiv 7 Dec 2023.

Builds on

57 of 87 references.

References · 57

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  5. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  6. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  7. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  8. 2023Segment Anythingcited by 8,158
  9. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  10. 2020End-to-End Object Detection with Transformerscited by 7,384
  11. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  12. 2020Scaling Laws for Neural Language Modelscited by 6,411
38 more
  1. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  2. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
  3. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  4. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  5. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  6. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,338
  7. 2021Learning to Prompt for Vision-Language Modelscited by 2,269
  8. 2022Visual Prompt Tuningcited by 1,674
  9. 2018BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learningcited by 1,282
  10. 2021BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-modelscited by 1,234
  11. 2021Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Conceptscited by 1,209
  12. 2021Per-Pixel Classification is Not All You Need for Semantic Segmentationcited by 1,148
  13. 2018ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Designcited by 1,132
  14. 2016Semantic Understanding of Scenes through the ADE20K Datasetcited by 1,063
  15. 2016Playing for Data: Ground Truth from Computer Gamescited by 816
  16. 2022Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distributioncited by 722
  17. 2022AdaptFormer: Adapting Vision Transformers for Scalable Visual Recognitioncited by 714
  18. 2022EVA: Exploring the Limits of Masked Visual Representation Learning at Scalecited by 685
  19. 2019Panoptic Feature Pyramid Networkscited by 644
  20. 2023EVA-02: A Visual Representation for Neon Genesiscited by 377
  21. 2018Two at Once: Enhancing Learning and Generalization Capacities via IBN-Netcited by 361
  22. 2022Mask DINO: Towards A Unified Transformer-based Framework for Object Detection and Segmentationcited by 341
  23. 2019Benchmarking Robustness in Object Detection: Autonomous Driving when Winter is Comingcited by 332
  24. 2019Domain Randomization and Pyramid Consistency: Simulation-to-Real Generalization without Accessing Target Domain Datacited by 245
  25. 2021RobustNet: Improving Domain Generalization in Urban-Scene Segmentation via Instance Selective Whiteningcited by 213
  26. 2019Switchable Whitening for Deep Representation Learningcited by 105
  27. 2022SegViT: Semantic Segmentation with Plain Vision Transformerscited by 105
  28. 2022Semantic-Aware Domain Generalized Segmentationcited by 78
  29. 2022WildNet: Learning Domain Generalized Semantic Segmentation from the Wildcited by 75
  30. 2022Adversarial Style Augmentation for Domain Generalized Urban-Scene Segmentationcited by 69
  31. 2021Global and Local Texture Randomization for Synthetic-to-Real Semantic Segmentationcited by 56
  32. 2022Pin the Memory: Learning to Generalize Semantic Segmentationcited by 48
  33. 2022PASTA: Proportional Amplitude Spectrum Training Augmentation for Syn-to-Real Domain Generalizationcited by 38
  34. 2021CrossNorm and SelfNorm for Generalization under Distribution Shiftscited by 33
  35. 2022Deliberated Domain Bridging for Domain Adaptive Semantic Segmentationcited by 31
  36. 2023HGFormer: Hierarchical Grouping Transformer for Domain Generalized Semantic Segmentationcited by 31
  37. 2023A Simple Recipe for Language-guided Domain Generalized Segmentationcited by 27
  38. 2021WEDGE: Web-Image Assisted Domain Generalization for Semantic Segmentationcited by 25

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 19; 2025: 43; 2026: 19.

82 papers.

Sort:

By area:

  1. 2024CrossEarth: Geospatial Vision Foundation Model for Domain Generalizable Remote Sensing Semantic Segmentationcited by 27
  2. 2025Advances in Multimodal Adaptation and Generalization: From Traditional Approaches to Foundation Modelscited by 23
  3. 2025VideoGen-Eval: Agent-based System for Video Generation Evaluationcited by 23
  4. 2025Mamba as a Bridge: Where Vision Foundation Models Meet Vision Language Models for Domain-Generalized Semantic Segmentationcited by 23
  5. 2024SoMA: Singular Value Decomposed Minor Components Adaptation for Domain Generalizable Representation Learningcited by 22
  6. 2024Learning Spectral-Decomposed Tokens for Domain Generalized Semantic Segmentationcited by 21
  7. 2024Textual Query-Driven Mask Transformer for Domain Generalized Segmentationcited by 20
  8. 2025Simplifying DINO via Coding Rate Regularizationcited by 20
  9. 2024How to Benchmark Vision Foundation Models for Semantic Segmentation?cited by 15
  10. 2025FisherTune: Fisher-Guided Robust Tuning of Vision Foundation Models for Domain Generalized Segmentationcited by 15
  11. 2024CoDA: Instructive Chain-of-Domain Adaptation with Severity-Aware Visual Prompt Tuningcited by 13
  12. 2025Learning Robust Stereo Matching in the Wild with Selective Mixture-of-Expertscited by 13
38 more
  1. 2023SkyScenes: A Synthetic Dataset for Aerial Scene Understandingcited by 12
  2. 2025A Large-Scale Referring Remote Sensing Image Segmentation Dataset and Benchmarkcited by 12
  3. 2024Exploring the Benefits of Vision Foundation Models for Unsupervised Domain Adaptationcited by 11
  4. 2025Large Self-Supervised Models Bridge the Gap in Domain Adaptive Object Detectioncited by 9
  5. 2025SANSA: Unleashing the Hidden Semantics in SAM2 for Few-Shot Segmentationcited by 8
  6. 2024Do Vision Foundation Models Enhance Domain Generalization in Medical Image Segmentation?cited by 6
  7. 2025Stronger, Steadier & Superior: Geometric Consistency in Depth VFM Forges Domain Generalized Semantic Segmentationcited by 6
  8. 2024Unsupervised Modality Adaptation with Text-to-Image Diffusion Models for Semantic Segmentationcited by 4
  9. 2025Exploiting Domain Properties in Language-Driven Domain Generalization for Semantic Segmentationcited by 4
  10. 2024Curriculum Fine-tuning of Vision Foundation Model for Medical Image Classification Under Label Noisecited by 3
  11. 2025Generalizable Multispectral Land Cover Classification via Frequency-Aware Mixture of Low-Rank Token Expertscited by 3
  12. 2025Exploring Probabilistic Modeling Beyond Domain Generalization for Semantic Segmentationcited by 3
  13. 2025DEAL-300K: Diffusion-based Editing Area Localization with a 300K-Scale Dataset and Frequency-Prompted Baselinecited by 3
  14. 2024FlowCLAS: Enhancing Normalizing Flow Via Contrastive Learning For Anomaly Segmentationcited by 2
  15. 2025SCHNet: SAM Marries CLIP for Human Parsingcited by 2
  16. 2025UCS: A Universal Model for Curvilinear Structure Segmentationcited by 2
  17. 2025Rein++: Efficient Generalization and Adaptation for Semantic Segmentation with Vision Foundation Modelscited by 2
  18. 2025ER-LoRA: Effective-Rank Guided Adaptation for Weather-Generalized Depth Estimationcited by 2
  19. 2025Improving Multimodal Distillation for 3D Semantic Segmentation under Domain Shiftcited by 2
  20. 2025Causal-Tune: Mining Causal Factors from Vision Foundation Models for Domain Generalized Semantic Segmentationcited by 2
  21. 2026Open-Vocabulary Domain Generalization in Urban-Scene Segmentationcited by 2
  22. 2026CrossEarth-SAR: A SAR-Centric and Billion-Scale Geospatial Foundation Model for Domain Generalizable Semantic Segmentationcited by 2
  23. 2024WeatherProof: Leveraging Language Guidance for Semantic Segmentation in Adverse Weathercited by 1
  24. 2024Transient Fault Tolerant Semantic Segmentation for Autonomous Drivingcited by 1
  25. 2024CLIP's Visual Embedding Projector is a Few-shot Cornucopiacited by 1
  26. 2024IV-tuning: Parameter-Efficient Transfer Learning for Infrared-Visible Taskscited by 1
  27. 2025MFM-DA: Instance-Aware Adaptor and Hierarchical Alignment for Efficient Domain Adaptation in Medical Foundation Modelscited by 1
  28. 2025VFM-UDA++: Improving Network Architectures and Data Strategies for Unsupervised Domain Adaptive Semantic Segmentationcited by 1
  29. 2025Earth-Adapter: Bridge the Geospatial Domain Gaps with Mixture of Frequency Adaptationcited by 1
  30. 2025What is the Added Value of UDA in the VFM Era?cited by 1
  31. 2025Leveraging Depth and Language for Open-Vocabulary Domain-Generalized Semantic Segmentationcited by 1
  32. 2025Prototypical Progressive Alignment and Reweighting for Generalizable Semantic Segmentationcited by 1
  33. 2025Towards Single-Source Domain Generalized Object Detection via Causal Visual Promptscited by 1
  34. 2025Advancing Autonomous Driving System Testing: Demands, Challenges, and Future Directionscited by 1
  35. 2025Toward Stable Semi-Supervised Remote Sensing Segmentation via Co-Guidance and Co-Fusioncited by 1
  36. 2026Local Precise Refinement: A Dual-Gated Mixture-of-Experts for Enhancing Foundation Model Generalization against Spectral Shiftscited by 1
  37. 2026SegFly: A Dataset and 2D-3D-2D Paradigm for Aerial RGB-Thermal Semantic Segmentation at Scalecited by 1
  38. 2024A Vanilla Multi-Task Framework for Dense Visual Prediction Solution to 1st VCL Challenge -- Multi-Task Robustness Trackcited by 0

50 most cited of 82 All 82 (JSON)