mlsanityData to 7 Oct 2026

Bridging Vision and Language Encoders: Parameter-Efficient Tuning for Referring Image Segmentation

arXiv 2307.11545 ↗First posted 21 Jul 2023cs.CV cs.CLcited by 61

Not on the map. Why?

Abstract (arXiv)

Parameter Efficient Tuning (PET) has gained attention for reducing the number of parameters while maintaining performance and providing better hardware resource savings, but few studies investigate dense prediction tasks and interaction between modalities. In this paper, we do an investigation of efficient tuning problems on referring image segmentation. We propose a novel adapter called Bridger to facilitate cross-modal information exchange and inject task-specific information into the pre-trained model. We also design a lightweight decoder for image segmentation. Our approach achieves comparable or superior performance with only 1.61% to 3.38% backbone parameter updates, evaluated on challenging benchmarks. The code is available at urlhttps://github.com/kkakkkka/ETRIS.

Key facts. Cited by 61 papers in the map (data through 7 Oct 2026). Builds on 41 of its 50 references that are in the map. First posted on arXiv 21 Jul 2023.

Builds on

41 of 50 references.

References · 41

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  6. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  7. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  8. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  9. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  10. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  11. 2021Learning to Prompt for Vision-Language Modelscited by 2,269
  12. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,219
29 more
  1. 2021BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-modelscited by 1,234
  2. 2020Oscar: Object-Semantics Aligned Pre-training for Vision-Language Taskscited by 1,213
  3. 2021Grounded Language-Image Pre-trainingcited by 1,147
  4. 2016Modeling Context in Referring Expressionscited by 1,054
  5. 2021CLIP-Adapter: Better Vision-Language Models with Feature Adapterscited by 1,021
  6. 2021MDETR -- Modulated Detection for End-to-End Multi-Modal Understandingcited by 811
  7. 2022PaLI: A Jointly-Scaled Multilingual Language-Image Modelcited by 762
  8. 2022AdaptFormer: Adapting Vision Transformers for Scalable Visual Recognitioncited by 714
  9. 2020Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuningcited by 658
  10. 2022Vision Transformer Adapter for Dense Predictionscited by 564
  11. 2021Compacter: Efficient Low-Rank Hypercomplex Adapter Layerscited by 458
  12. 2018MAttNet: Modular Attention Network for Referring Expression Comprehensioncited by 427
  13. 2021Early Convolutions Help Transformers See Bettercited by 395
  14. 2021CRIS: CLIP-Driven Referring Image Segmentationcited by 353
  15. 2020Parameter-Efficient Transfer Learning with Diff Pruningcited by 351
  16. 2021VL-Adapter: Parameter-Efficient Transfer Learning for Vision-and-Language Taskscited by 319
  17. 2019Cross-Modal Self-Attention Network for Referring Image Segmentationcited by 263
  18. 2019UNITER: UNiversal Image-TExt Representation Learningcited by 225
  19. 2020Referring Image Segmentation via Cross-Modal Progressive Comprehensioncited by 131
  20. 2017Recurrent Multimodal Interaction for Referring Image Segmentationcited by 125
  21. 2022ReSTR: Convolution-free Referring Image Segmentation Using Transformerscited by 112
  22. 2021Encoder Fusion Network with Co-Attention Embedding for Referring Image Segmentationcited by 111
  23. 2022VLT: Vision-Language Transformer and Query Generation for Referring Segmentationcited by 110
  24. 2022Multi-Modal Masked Autoencoders for Medical Vision-and-Language Pre-Trainingcited by 104
  25. 2020Linguistic Structure Guided Context Modeling for Referring Image Segmentationcited by 102
  26. 2021Cross-Modal Progressive Comprehension for Referring Segmentationcited by 102
  27. 2022Align, Reason and Learn: Enhancing Medical Vision-and-Language Pre-training with Knowledgecited by 51
  28. 2019Referring Expression Object Segmentation with Caption-Aware Consistencycited by 46
  29. 2022Position-Aware Contrastive Alignment for Referring Image Segmentationcited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 6; 2024: 30; 2025: 13; 2026: 12.

61 papers.

Sort:

By area:

  1. 2024Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Surveycited by 641
  2. 2023Parameter-Efficient Fine-Tuning Methods for Pretrained Language Models: A Critical Review and Assessmentcited by 227
  3. 2024SAM4MLLM: Enhance Multi-Modal Large Language Model for Referring Expression Segmentationcited by 68
  4. 2023Consistent123: One Image to Highly Consistent 3D Asset Using Case-Aware Diffusion Priorscited by 64
  5. 2023Strategic Preys Make Acute Predators: Enhancing Camouflaged Object Detectors by Generating Camouflaged Objectscited by 56
  6. 2024Explainable and Interpretable Multimodal Large Language Models: A Comprehensive Surveycited by 52
  7. 2024ClipSAM: CLIP and SAM Collaboration for Zero-Shot Anomaly Segmentationcited by 49
  8. 2024MambaTalk: Efficient Holistic Gesture Synthesis with Selective State Space Modelscited by 46
  9. 2024SAMWISE: Infusing Wisdom in SAM2 for Text-Driven Video Segmentationcited by 43
  10. 2025Parameter-Efficient Fine-Tuning for Foundation Modelscited by 43
  11. 2024ReMamber: Referring Image Segmentation with Mamba Twistercited by 38
  12. 2024Image Segmentation in Foundation Model Era: A Surveycited by 35
38 more
  1. 2025FireEdit: Fine-grained Instruction-based Image Editing via Region-aware Vision Language Modelcited by 31
  2. 2025SAM-R1: Leveraging SAM for Reward Feedback in Multimodal Segmentation via Reinforcement Learningcited by 31
  3. 2024Real-world Image Dehazing with Coherence-based Pseudo Labeling and Cooperative Unfolding Networkcited by 26
  4. 2024Cross-Modal Bidirectional Interaction Model for Referring Remote Sensing Image Segmentationcited by 26
  5. 2024Curriculum Point Prompting for Weakly-Supervised Referring Image Segmentationcited by 23
  6. 2025Multimodal Referring Segmentation: A Surveycited by 22
  7. 2023Chain of Generation: Multi-Modal Gesture Synthesis via Cascaded Conditional Controlcited by 18
  8. 2024Test-Time Low Rank Adaptation via Confidence Maximization for Zero-Shot Generalization of Vision-Language Modelscited by 17
  9. 2025RSRefSeg 2: Decoupling Referring Remote Sensing Image Segmentation with Foundation Modelscited by 17
  10. 2024Show Me What and Where has Changed? Question Answering and Grounding for Remote Sensing Change Detectioncited by 15
  11. 2024Pseudo-RIS: Distinctive Pseudo-supervision Generation for Referring Image Segmentationcited by 13
  12. 2024Sparse-Tuning: Adapting Vision Transformers with Efficient Fine-tuning and Inferencecited by 12
  13. 2024RESMatch: Referring Expression Segmentation in a Semi-Supervised Mannercited by 11
  14. 2024M2IST: Multi-Modal Interactive Side-Tuning for Efficient Referring Expression Comprehensioncited by 11
  15. 2024Alignment is All You Need: A Training-free Augmentation Strategy for Pose-guided Video Generationcited by 10
  16. 2024What does CLIP know about peeling a banana?cited by 9
  17. 2024DreamDissector: Learning Disentangled Text-to-3D Generation from 2D Diffusion Priorscited by 8
  18. 2025Free on the Fly: Enhancing Flexibility in Test-Time Adaptation with Online EMcited by 8
  19. 2024Vision-Aware Text Features in Referring Image Segmentation: From Object Understanding to Context Understandingcited by 7
  20. 2024R2-Bench: Benchmarking the Robustness of Referring Perception Models under Perturbationscited by 6
  21. 2024MaPPER: Multimodal Prior-guided Parameter Efficient Tuning for Referring Expression Comprehensioncited by 6
  22. 2025DiffRIS: Enhancing Referring Remote Sensing Image Segmentation with Pre-trained Text-to-Image Diffusion Modelscited by 6
  23. 2025Alleviating Textual Reliance in Medical Language-guided Segmentation via Prototype-driven Semantic Approximationcited by 6
  24. 2024DARA: Domain- and Relation-aware Adapters Make Parameter-efficient Tuning for Visual Groundingcited by 5
  25. 2026CroBIM-U: Uncertainty-Driven Referring Remote Sensing Image Segmentationcited by 5
  26. 2026GREx: Generalized Referring Expression Segmentation, Comprehension, and Generationcited by 5
  27. 2024Deep Instruction Tuning for Segment Anything Modelcited by 4
  28. 2024PTQ4RIS: Post-Training Quantization for Referring Image Segmentationcited by 4
  29. 2025Segment Anyword: Mask Prompt Inversion for Open-Set Grounded Segmentationcited by 4
  30. 2024A Simple Baseline with Single-encoder for Referring Image Segmentationcited by 2
  31. 2024MaskRIS: Semantic Distortion-aware Data Augmentation for Referring Image Segmentationcited by 2
  32. 2025Referring Remote Sensing Image Segmentation with Cross-view Semantics Interaction Networkcited by 2
  33. 2023Synchronizing Vision and Language: Bidirectional Token-Masking AutoEncoder for Referring Image Segmentationcited by 1
  34. 2024Harnessing Vision-Language Pretrained Models with Temporal-Aware Adaptation for Referring Video Object Segmentationcited by 1
  35. 2025ZING-3D: Zero-shot Incremental 3D Scene Graphs via Vision-Language Modelscited by 1
  36. 2026TALENT: Target-aware Efficient Tuning for Referring Image Segmentationcited by 1
  37. 2023Towards Generalizable Referring Image Segmentation via Target Prompt and Visual Coherencecited by 0
  38. 2024Phrase-Instance Alignment for Generalized Referring Segmentationcited by 0

50 most cited of 61 All 61 (JSON)