# Sewer-ML: A Multi-Label Sewer Defect Classification Dataset and Benchmark

arXiv: https://arxiv.org/abs/2103.10895 · HTML: https://mlsanity.com/paper/2103.10895/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 92 references that are in the map.
- First posted on arXiv 19 Mar 2021.

## Builds on (most cited first, 30 of 36)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour](https://mlsanity.com/paper/1706.02677/) (2017)
- [Class-Balanced Loss Based on Effective Number of Samples](https://mlsanity.com/paper/1901.05555/) (2019)
- [SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size](https://mlsanity.com/paper/1602.07360/) (2016)
- [Focal Loss for Dense Object Detection](https://mlsanity.com/paper/1708.02002/) (2017)
- [The Open Images Dataset V4: Unified image classification, object detection, and visual relationship detection at scale](https://mlsanity.com/paper/1811.00982/) (2018)
- [Asymmetric Loss For Multi-Label Classification](https://mlsanity.com/paper/2009.14119/) (2020)
- [Multi-Label Image Recognition with Graph Convolutional Networks](https://mlsanity.com/paper/1904.03582/) (2019)
- [CNN-RNN: A Unified Framework for Multi-label Image Classification](https://mlsanity.com/paper/1604.04573/) (2016)
- [Learning a Deep ConvNet for Multi-label Classification with Partial Labels](https://mlsanity.com/paper/1902.09720/) (2019)
- [Distribution-Balanced Loss for Multi-Label Classification in Long-Tailed Datasets](https://mlsanity.com/paper/2007.09654/) (2020)
- [TResNet: High Performance GPU-Dedicated Architecture](https://mlsanity.com/paper/2003.13630/) (2020)
- [Learning Semantic-Specific Graph Representation for Multi-Label Image Recognition](https://mlsanity.com/paper/1908.07325/) (2019)
- [Learning Spatial Regularization with Image-level Supervisions for Multi-label Image Classification](https://mlsanity.com/paper/1702.05891/) (2017)
- [Multi-label Image Recognition by Recurrently Discovering Attentional Regions](https://mlsanity.com/paper/1711.02816/) (2017)
- [Cross-Modality Attention with Semantic Graph Embedding for Multi-Label Classification](https://mlsanity.com/paper/1912.07872/) (2019)
- [Multi-Label Zero-Shot Learning with Structured Knowledge Graphs](https://mlsanity.com/paper/1711.06526/) (2017)
- [Rosetta: Large scale system for text detection and recognition in images](https://mlsanity.com/paper/1910.05085/) (2019)
- [Multi-Label Classification with Label Graph Superimposing](https://mlsanity.com/paper/1911.09243/) (2019)
- [CNN: Single-label to Multi-label](https://mlsanity.com/paper/1406.5726/) (2014)
- [Deep Convolutional Ranking for Multilabel Image Annotation](https://mlsanity.com/paper/1312.4894/) (2013)
- [Orderless Recurrent Models for Multi-label Classification](https://mlsanity.com/paper/1911.09996/) (2019)
- [Recurrent Attentional Reinforcement Learning for Multi-label Image Recognition](https://mlsanity.com/paper/1712.07465/) (2017)
- [Improving Pairwise Ranking for Multi-label Image Classification](https://mlsanity.com/paper/1704.03135/) (2017)

## Built on by (most cited first, 17 of 17)

- [Random Word Data Augmentation with CLIP for Zero-Shot Anomaly Detection](https://mlsanity.com/paper/2308.11119/) (2023)
- [GrainSpace: A Large-scale Dataset for Fine-grained and Domain-adaptive Recognition of Cereal Grains](https://mlsanity.com/paper/2203.05306/) (2022)
- [Dual Attention U-Net with Feature Infusion: Pushing the Boundaries of Multiclass Defect Segmentation](https://mlsanity.com/paper/2312.14053/) (2023)
- [Towards Trustworthy Multi-label Sewer Defect Classification via Evidential Deep Learning](https://mlsanity.com/paper/2210.13782/) (2022)
- [VideoPipe 2022 Challenge: Real-World Video Understanding for Urban Pipe Inspection](https://mlsanity.com/paper/2210.11158/) (2022)
- [Multi-Task Classification of Sewer Pipe Defects and Properties using a Cross-Task Graph Neural Network Decoder](https://mlsanity.com/paper/2111.07846/) (2021)
- [MN-Pair Contrastive Damage Representation and Clustering for Prognostic Explanation](https://mlsanity.com/paper/2301.06077/) (2023)
- [OpenTrench3D: A Photogrammetric 3D Point Cloud Dataset for Semantic Segmentation of Underground Utilities](https://mlsanity.com/paper/2404.07711/) (2024)
- [Progressive Alignment with VLM-LLM Feature to Augment Defect Classification for the ASE Dataset](https://mlsanity.com/paper/2404.05183/) (2024)
- [Sewer Image Super-Resolution with Depth Priors and Its Lightweight Network](https://mlsanity.com/paper/2407.19271/) (2024)
- [Multi-label Sewer Pipe Defect Recognition with Mask Attention Feature Enhancement and Label Correlation Learning](https://mlsanity.com/paper/2408.00489/) (2024)
- [Self-Supervised Learning for Identifying Defects in Sewer Footage](https://mlsanity.com/paper/2409.02140/) (2024)
- [Self-Supervised Anomaly Detection in the Wild: Favor Joint Embeddings Methods](https://mlsanity.com/paper/2410.04289/) (2024)
- [On The Relationship between Visual Anomaly-free and Anomalous Representations](https://mlsanity.com/paper/2410.06576/) (2024)
- [When the Small-Loss Trick is Not Enough: Multi-Label Image Classification with Noisy Labels Applied to CCTV Sewer Inspections](https://mlsanity.com/paper/2410.07689/) (2024)
- [Any-Class Presence Likelihood for Robust Multi-Label Classification with Abundant Negative Data](https://mlsanity.com/paper/2506.05721/) (2025)
- [Vision Transformer-Based Multi-Level Feature Fusion for Multi-Label Sewer Defect Classification](https://mlsanity.com/paper/2609.11375/) (2026)

## Abstract (arXiv)

Perhaps surprisingly sewerage infrastructure is one of the most costly infrastructures in modern society. Sewer pipes are manually inspected to determine whether the pipes are defective. However, this process is limited by the number of qualified inspectors and the time it takes to inspect a pipe. Automatization of this process is therefore of high interest. So far, the success of computer vision approaches for sewer defect classification has been limited when compared to the success in other fields mainly due to the lack of public datasets. To this end, in this work we present a large novel and publicly available multi-label classification dataset for image-based sewer defect classification called Sewer-ML. The Sewer-ML dataset consists of 1.3 million images annotated by professional sewer inspectors from three different utility companies across nine years. Together with the dataset, we also present a benchmark algorithm and a novel metric for assessing performance. The benchmark algorithm is a result of evaluating 12 state-of-the-art algorithms, six from the sewer defect classification domain and six from the multi-label classification domain, and combining the best performing algorithms. The novel metric is a class-importance weighted F2 score, F2CIW, reflecting the economic impact of each class, used together with the normal pipe F1 score, F1Normal. The benchmark algorithm achieves an F2CIW score of 55.11% and F1Normal score of 90.94%, leaving ample room for improvement on the Sewer-ML dataset. The code, models, and dataset are available at the project page https://vap.aau.dk/sewer-ml/