# PCSR: Pseudo-label Consistency-Guided Sample Refinement for Noisy Correspondence Learning

arXiv: https://arxiv.org/abs/2509.15623 · HTML: https://mlsanity.com/paper/2509.15623/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 38 references that are in the map.
- First posted on arXiv 19 Sep 2025.

## Builds on (most cited first, 18 of 18)

- [Sigmoid Loss for Language Image Pre-Training](https://mlsanity.com/paper/2303.15343/) (2023)
- [Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels](https://mlsanity.com/paper/1805.07836/) (2018)
- [Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning](https://mlsanity.com/paper/1704.03976/) (2017)
- [Stacked Cross Attention for Image-Text Matching](https://mlsanity.com/paper/1803.08024/) (2018)
- [Visual Semantic Reasoning for Image-Text Matching](https://mlsanity.com/paper/1909.02701/) (2019)
- [Learning the Best Pooling Strategy for Visual Semantic Embedding](https://mlsanity.com/paper/2011.04305/) (2020)
- [Similarity Reasoning and Filtration for Image-Text Matching](https://mlsanity.com/paper/2101.01368/) (2021)
- [Understanding and Improving Early Stopping for Learning with Noisy Labels](https://mlsanity.com/paper/2106.15853/) (2021)
- [Graph Structured Network for Image-Text Matching](https://mlsanity.com/paper/2004.00277/) (2020)
- [Cross-modal Scene Graph Matching for Relationship-aware Image-Text Retrieval](https://mlsanity.com/paper/1910.05134/) (2019)
- [Learning with Neighbor Consistency for Noisy Labels](https://mlsanity.com/paper/2202.02200/) (2022)
- [Robust Contrastive Learning against Noisy Views](https://mlsanity.com/paper/2201.04309/) (2022)
- [Preparing Lessons: Improve Knowledge Distillation with Better Supervision](https://mlsanity.com/paper/1911.07471/) (2019)
- [BiCro: Noisy Correspondence Rectification for Multi-modality Data via Bi-directional Cross-modal Similarity Consistency](https://mlsanity.com/paper/2303.12419/) (2023)
- [Noisy Correspondence Learning with Meta Similarity Correction](https://mlsanity.com/paper/2304.06275/) (2023)
- [Tackling Instance-Dependent Label Noise via a Universal Probabilistic Model](https://mlsanity.com/paper/2101.05467/) (2021)
- [Asymmetric Visual Semantic Embedding Framework for Efficient Vision-Language Alignment](https://mlsanity.com/paper/2503.06974/) (2025)
- [Fast FixMatch: Faster Semi-Supervised Learning with Curriculum Batch Size](https://mlsanity.com/paper/2309.03469/) (2023)

## Built on by (most cited first, 1 of 1)

- [Intra-Modal Neighbors Never Lie: Rectifying Inter-Modal Noisy Correspondence via Graph-Based Intra-Modal Reasoning](https://mlsanity.com/paper/2606.04061/) (2026)

## Abstract (arXiv)

Cross-modal retrieval aims to align different modalities via semantic similarity. However, existing methods often assume that image-text pairs are perfectly aligned, overlooking Noisy Correspondences in real data. These misaligned pairs misguide similarity learning and degrade retrieval performance. Previous methods often rely on coarse-grained categorizations that simply divide data into clean and noisy samples, overlooking the intrinsic diversity within noisy instances. Moreover, they typically apply uniform training strategies regardless of sample characteristics, resulting in suboptimal sample utilization for model optimization. To address the above challenges, we introduce a novel framework, called Pseudo-label Consistency-Guided Sample Refinement (PCSR), which enhances correspondence reliability by explicitly dividing samples based on pseudo-label consistency. Specifically, we first employ a confidence-based estimation to distinguish clean and noisy pairs, then refine the noisy pairs via pseudo-label consistency to uncover structurally distinct subsets. We further proposed a Pseudo-label Consistency Score (PCS) to quantify prediction stability, enabling the separation of ambiguous and refinable samples within noisy pairs. Accordingly, we adopt Adaptive Pair Optimization (APO), where ambiguous samples are optimized with robust loss functions and refinable ones are enhanced via text replacement during training. Extensive experiments on CC152K, MS-COCO and Flickr30K validate the effectiveness of our method in improving retrieval robustness under noisy supervision.