# BiCro: Noisy Correspondence Rectification for Multi-modality Data via Bi-directional Cross-modal Similarity Consistency

arXiv: https://arxiv.org/abs/2303.12419 · HTML: https://mlsanity.com/paper/2303.12419/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 58 references that are in the map.
- First posted on arXiv 22 Mar 2023.

## Builds on (most cited first, 30 of 36)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Understanding deep learning requires rethinking generalization](https://mlsanity.com/paper/1611.03530/) (2016)
- [Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering](https://mlsanity.com/paper/1707.07998/) (2017)
- [ViLT: Vision-and-Language Transformer Without Convolution or Region Supervision](https://mlsanity.com/paper/2102.03334/) (2021)
- [Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labels](https://mlsanity.com/paper/1804.06872/) (2018)
- [DivideMix: Learning with Noisy Labels as Semi-supervised Learning](https://mlsanity.com/paper/2002.07394/) (2020)
- [Learning to Reweight Examples for Robust Deep Learning](https://mlsanity.com/paper/1803.09050/) (2018)
- [Making Deep Neural Networks Robust to Label Noise: a Loss Correction Approach](https://mlsanity.com/paper/1609.03683/) (2016)
- [MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labels](https://mlsanity.com/paper/1712.05055/) (2017)
- [Stacked Cross Attention for Image-Text Matching](https://mlsanity.com/paper/1803.08024/) (2018)
- [Unsupervised Label Noise Modeling and Loss Correction](https://mlsanity.com/paper/1904.11238/) (2019)
- [Training Deep Neural Networks on Noisy Labels with Bootstrapping](https://mlsanity.com/paper/1412.6596/) (2014)
- [Joint Optimization Framework for Learning with Noisy Labels](https://mlsanity.com/paper/1803.11364/) (2018)
- [Normalized Loss Functions for Deep Learning with Noisy Labels](https://mlsanity.com/paper/2006.13554/) (2020)
- [Decoupling "when to update" from "how to update"](https://mlsanity.com/paper/1706.02613/) (2017)
- [Are Anchor Points Really Indispensable in Label-Noise Learning?](https://mlsanity.com/paper/1906.00189/) (2019)
- [Visual Semantic Reasoning for Image-Text Matching](https://mlsanity.com/paper/1909.02701/) (2019)
- [Dimensionality-Driven Learning with Noisy Labels](https://mlsanity.com/paper/1806.02612/) (2018)
- [Learning from Noisy Labels with Distillation](https://mlsanity.com/paper/1703.02391/) (2017)
- [Gradient Descent with Early Stopping is Provably Robust to Label Noise for Overparameterized Neural Networks](https://mlsanity.com/paper/1903.11680/) (2019)
- [CurriculumNet: Weakly Supervised Learning from Large-Scale Web Images](https://mlsanity.com/paper/1808.01097/) (2018)
- [Similarity Reasoning and Filtration for Image-Text Matching](https://mlsanity.com/paper/2101.01368/) (2021)
- [Learning Two-Branch Neural Networks for Image-Text Matching Tasks](https://mlsanity.com/paper/1704.03470/) (2017)
- [Learning From Noisy Large-Scale Datasets With Minimal Supervision](https://mlsanity.com/paper/1701.01619/) (2017)
- [IMRAM: Iterative Matching with Recurrent Attention Memory for Cross-Modal Image-Text Retrieval](https://mlsanity.com/paper/2003.03772/) (2020)
- [Masking: A New Perspective of Noisy Supervision](https://mlsanity.com/paper/1805.08193/) (2018)
- [Sample Selection with Uncertainty of Losses for Learning with Noisy Labels](https://mlsanity.com/paper/2106.00445/) (2021)
- [Toward Robustness against Label Noise in Training Deep Discriminative Neural Networks](https://mlsanity.com/paper/1706.00038/) (2017)
- [Error-Bounded Correction of Noisy Labels](https://mlsanity.com/paper/2011.10077/) (2020)

## Built on by (most cited first, 27 of 27)

- [Multi-granularity Correspondence Learning from Long-term Noisy Videos](https://mlsanity.com/paper/2401.16702/) (2024)
- [Cross-modal Active Complementary Learning with Self-refining Correspondence](https://mlsanity.com/paper/2310.17468/) (2023)
- [Continual Multimodal Contrastive Learning](https://mlsanity.com/paper/2503.14963/) (2025)
- [IRGPT: Understanding Real-world Infrared Image with Bi-cross-modal Curriculum on Large-scale Benchmark](https://mlsanity.com/paper/2507.14449/) (2025)
- [Learning to Rematch Mismatched Pairs for Robust Cross-Modal Retrieval](https://mlsanity.com/paper/2403.05105/) (2024)
- [Mitigating Noisy Correspondence by Geometrical Structure Consistency Learning](https://mlsanity.com/paper/2405.16996/) (2024)
- [Negative Pre-aware for Noisy Cross-modal Matching](https://mlsanity.com/paper/2312.05777/) (2023)
- [Noisy Correspondence Learning with Self-Reinforcing Errors Mitigation](https://mlsanity.com/paper/2312.16478/) (2023)
- [Fewer Steps, Better Performance: Efficient Cross-Modal Clip Trimming for Video Moment Retrieval Using Language](https://mlsanity.com/paper/2605.29793/) (2026)
- [Disentangled Noisy Correspondence Learning](https://mlsanity.com/paper/2408.05503/) (2024)
- [CAPro: Webly Supervised Learning with Cross-Modality Aligned Prototypes](https://mlsanity.com/paper/2310.09761/) (2023)
- [ReCon: Enhancing True Correspondence Discrimination through Relation Consistency for Robust Noisy Correspondence Learning](https://mlsanity.com/paper/2502.19962/) (2025)
- [Differential-informed Sample Selection Accelerates Multimodal Contrastive Learning](https://mlsanity.com/paper/2507.12998/) (2025)
- [Composed Multi-modal Retrieval: A Survey of Approaches and Applications](https://mlsanity.com/paper/2503.01334/) (2025)
- [UtilGen: Utility-Centric Generative Data Augmentation with Dual-Level Task Adaptation](https://mlsanity.com/paper/2510.24262/) (2025)
- [TSVC:Tripartite Learning with Semantic Variation Consistency for Robust Image-Text Retrieval](https://mlsanity.com/paper/2501.10935/) (2025)
- [iEBAKER: Improved Remote Sensing Image-Text Retrieval Framework via Eliminate Before Align and Keyword Explicit Reasoning](https://mlsanity.com/paper/2504.05644/) (2025)
- [Multi-Modal Dataset Distillation in the Wild](https://mlsanity.com/paper/2506.01586/) (2025)
- [PCSR: Pseudo-label Consistency-Guided Sample Refinement for Noisy Correspondence Learning](https://mlsanity.com/paper/2509.15623/) (2025)
- [Variational Adapter for Cross-modal Similarity Representation](https://mlsanity.com/paper/2605.30968/) (2026)
- [REPAIR: Rank Correlation and Noisy Pair Half-replacing with Memory for Noisy Correspondence](https://mlsanity.com/paper/2403.08224/) (2024)
- [Debiased Orthogonal Boundary-Driven Efficient Noise Mitigation](https://mlsanity.com/paper/2410.01944/) (2024)
- [Robust Noisy Correspondence Learning via Self-Drop and Dual-Weight](https://mlsanity.com/paper/2412.06172/) (2024)
- [PAUL: Uncertainty-Guided Partition and Augmentation for Robust Cross-View Geo-Localization under Noisy Correspondence](https://mlsanity.com/paper/2508.20066/) (2025)
- [Testing chatbots on the creation of encoders for audio conditioned image generation](https://mlsanity.com/paper/2509.09717/) (2025)
- [PMPGuard: Catching Pseudo-Matched Pairs in Remote Sensing Image-Text Retrieval](https://mlsanity.com/paper/2512.18660/) (2025)
- [Intra-Modal Neighbors Never Lie: Rectifying Inter-Modal Noisy Correspondence via Graph-Based Intra-Modal Reasoning](https://mlsanity.com/paper/2606.04061/) (2026)

## Abstract (arXiv)

As one of the most fundamental techniques in multimodal learning, cross-modal matching aims to project various sensory modalities into a shared feature space. To achieve this, massive and correctly aligned data pairs are required for model training. However, unlike unimodal datasets, multimodal datasets are extremely harder to collect and annotate precisely. As an alternative, the co-occurred data pairs (e.g., image-text pairs) collected from the Internet have been widely exploited in the area. Unfortunately, the cheaply collected dataset unavoidably contains many mismatched data pairs, which have been proven to be harmful to the model's performance. To address this, we propose a general framework called BiCro (Bidirectional Cross-modal similarity consistency), which can be easily integrated into existing cross-modal matching models and improve their robustness against noisy data. Specifically, BiCro aims to estimate soft labels for noisy data pairs to reflect their true correspondence degree. The basic idea of BiCro is motivated by that -- taking image-text matching as an example -- similar images should have similar textual descriptions and vice versa. Then the consistency of these two similarities can be recast as the estimated soft labels to train the matching model. The experiments on three popular cross-modal matching datasets demonstrate that our method significantly improves the noise-robustness of various matching models, and surpass the state-of-the-art by a clear margin.