# Weakly Supervised Learning with Side Information for Noisy Labeled Images

arXiv: https://arxiv.org/abs/2008.11586 · HTML: https://mlsanity.com/paper/2008.11586/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Aug 2020.

## Built on by (most cited first, 30 of 30)

- [The Met Dataset: Instance-level Recognition for Artworks](https://mlsanity.com/paper/2202.01747/) (2022)
- [RIME: Robust Preference-based Reinforcement Learning with Noisy Preferences](https://mlsanity.com/paper/2402.17257/) (2024)
- [Noise-aware Learning from Web-crawled Image-Text Data for Image Captioning](https://mlsanity.com/paper/2212.13563/) (2022)
- [ARMBench: An Object-centric Benchmark Dataset for Robotic Manipulation](https://mlsanity.com/paper/2303.16382/) (2023)
- [Instance-Dependent Noisy Label Learning via Graphical Modelling](https://mlsanity.com/paper/2209.00906/) (2022)
- [Don't drop your samples! Coherence-aware training benefits Conditional diffusion](https://mlsanity.com/paper/2405.20324/) (2024)
- [Learning Discriminative Dynamics with Label Corruption for Noisy Label Detection](https://mlsanity.com/paper/2405.19902/) (2024)
- [Skeleton-Based Human Action Recognition with Noisy Labels](https://mlsanity.com/paper/2403.09975/) (2024)
- [Learning from Noisy Labels via Dynamic Loss Thresholding](https://mlsanity.com/paper/2104.02570/) (2021)
- [A Light Touch Approach to Teaching Transformers Multi-view Geometry](https://mlsanity.com/paper/2211.15107/) (2022)
- [Mitigating Label Noise using Prompt-Based Hyperbolic Meta-Learning in Open-Set Domain Generalization](https://mlsanity.com/paper/2412.18342/) (2024)
- [Same or Not? Enhancing Visual Perception in Vision-Language Models](https://mlsanity.com/paper/2512.23592/) (2025)
- [Breast Cancer Molecular Subtypes Prediction on Pathological Images with Discriminative Patch Selecting and Multi-Instance Learning](https://mlsanity.com/paper/2203.07659/) (2022)
- [V2L: Leveraging Vision and Vision-language Models into Large-scale Product Retrieval](https://mlsanity.com/paper/2207.12994/) (2022)
- [CAPro: Webly Supervised Learning with Cross-Modality Aligned Prototypes](https://mlsanity.com/paper/2310.09761/) (2023)
- [PP-ShiTu: A Practical Lightweight Image Recognition System](https://mlsanity.com/paper/2111.00775/) (2021)
- [Unitail: Detecting, Reading, and Matching in Retail Scene](https://mlsanity.com/paper/2204.00298/) (2022)
- [Retail-786k: a Large-Scale Dataset for Visual Entity Matching](https://mlsanity.com/paper/2309.17164/) (2023)
- [The Capabilities and Limitations of Weak-to-Strong Generalization: Generalization and Calibration](https://mlsanity.com/paper/2502.01458/) (2025)
- [FoPro: Few-Shot Guided Robust Webly-Supervised Prototypical Learning](https://mlsanity.com/paper/2212.00465/) (2022)
- [PASS: Peer-Agreement based Sample Selection for training with Noisy Labels](https://mlsanity.com/paper/2303.10802/) (2023)
- [Weakly-semi-supervised object detection in remotely sensed imagery](https://mlsanity.com/paper/2311.17449/) (2023)
- [Predicting Perceived Gloss: Do Weak Labels Suffice?](https://mlsanity.com/paper/2403.17672/) (2024)
- [EReLiFM: Evidential Reliability-Aware Residual Flow Meta-Learning for Open-Set Domain Generalization under Noisy Labels](https://mlsanity.com/paper/2510.12687/) (2025)
- [Solution for Large-scale Long-tailed Recognition with Noisy Labels](https://mlsanity.com/paper/2106.10683/) (2021)
- [XLDA: Linear Discriminant Analysis for Scaling Continual Learning to Extreme Classification at the Edge](https://mlsanity.com/paper/2307.11317/) (2023)
- [Resist Label Noise with PGM for Graph Neural Networks](https://mlsanity.com/paper/2311.02116/) (2023)
- [MoMBS: Mixed-order minibatch sampling enhances model training from diverse-quality images](https://mlsanity.com/paper/2505.18741/) (2025)
- [Learning from M-Tuple Dominant Positive and Unlabeled Data](https://mlsanity.com/paper/2506.15686/) (2025)
- [On Revisiting Entropy for Identifying Mislabeled Images](https://mlsanity.com/paper/2605.31090/) (2026)

## Abstract (arXiv)

In many real-world datasets, like WebVision, the performance of DNN based classifier is often limited by the noisy labeled data. To tackle this problem, some image related side information, such as captions and tags, often reveal underlying relationships across images. In this paper, we present an efficient weakly supervised learning by using a Side Information Network (SINet), which aims to effectively carry out a large scale classification with severely noisy labels. The proposed SINet consists of a visual prototype module and a noise weighting module. The visual prototype module is designed to generate a compact representation for each category by introducing the side information. The noise weighting module aims to estimate the correctness of each noisy image and produce a confidence score for image ranking during the training procedure. The propsed SINet can largely alleviate the negative impact of noisy image labels, and is beneficial to train a high performance CNN based classifier. Besides, we released a fine-grained product dataset called AliProducts, which contains more than 2.5 million noisy web images crawled from the internet by using queries generated from 50,000 fine-grained semantic classes. Extensive experiments on several popular benchmarks (i.e. Webvision, ImageNet and Clothing-1M) and our proposed AliProducts achieve state-of-the-art performance. The SINet has won the first place in the classification task on WebVision Challenge 2019, and outperformed other competitors by a large margin.