# CLARiTy: A Vision Transformer for Multi-Label Classification and Weakly-Supervised Localization of Chest X-ray Pathologies

arXiv: https://arxiv.org/abs/2512.16700 · HTML: https://mlsanity.com/paper/2512.16700/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 74 references that are in the map.
- First posted on arXiv 18 Dec 2025.

## Builds on (most cited first, 30 of 37)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison](https://mlsanity.com/paper/1901.07031/) (2019)
- [ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseases](https://mlsanity.com/paper/1705.02315/) (2017)
- [PadChest: A large chest x-ray image dataset with multi-label annotated reports](https://mlsanity.com/paper/1901.07441/) (2019)
- [VinDr-CXR: An open dataset of chest X-rays with radiologist's annotations](https://mlsanity.com/paper/2012.15029/) (2020)
- [TinyViT: Fast Pretraining Distillation for Small Vision Transformers](https://mlsanity.com/paper/2207.10666/) (2022)
- [TieNet: Text-Image Embedding Network for Common Thorax Disease Classification and Reporting in Chest X-rays](https://mlsanity.com/paper/1801.04334/) (2018)
- [TorchXRayVision: A library of chest X-ray datasets and models](https://mlsanity.com/paper/2111.00595/) (2021)
- [Deep Learning for Chest X-ray Analysis: A Survey](https://mlsanity.com/paper/2103.08700/) (2021)
- [Thoracic Disease Identification and Localization with Limited Supervision](https://mlsanity.com/paper/1711.06373/) (2017)
- [MobileDets: Searching for Object Detection Architectures for Mobile Accelerators](https://mlsanity.com/paper/2004.14525/) (2020)
- [Reliable Tuberculosis Detection using Chest X-ray with Deep Learning, Segmentation and Visualization](https://mlsanity.com/paper/2007.14895/) (2020)
- [The Clever Hans Mirage: A Comprehensive Survey on Spurious Correlations in Machine Learning](https://mlsanity.com/paper/2402.12715/) (2024)
- [Learning to diagnose from scratch by exploiting dependencies among labels](https://mlsanity.com/paper/1710.10501/) (2017)
- [Learning to recognize Abnormalities in Chest X-Rays with Location-Aware Dense Networks](https://mlsanity.com/paper/1803.04565/) (2018)
- [Attention-Guided Curriculum Learning for Weakly Supervised Classification and Localization of Thoracic Diseases on Chest Radiographs](https://mlsanity.com/paper/1807.07532/) (2018)
- [Improving deep neural network generalization and robustness to background bias via layer-wise relevance propagation optimization](https://mlsanity.com/paper/2202.00232/) (2022)
- [Learning Hierarchical Attention for Weakly-supervised Chest X-Ray Abnormality Localization and Diagnosis](https://mlsanity.com/paper/2112.12349/) (2021)
- [Weakly Supervised Medical Diagnosis and Localization from Multiple Resolutions](https://mlsanity.com/paper/1803.07703/) (2018)
- [Multi-label Thoracic Disease Image Classification with Cross-Attention Networks](https://mlsanity.com/paper/2007.10859/) (2020)
- [AnaXNet: Anatomy Aware Multi-label Finding Classification in Chest X-ray](https://mlsanity.com/paper/2105.09937/) (2021)
- [Background Activation Suppression for Weakly Supervised Object Localization and Semantic Segmentation](https://mlsanity.com/paper/2309.12943/) (2023)
- [Weakly-Supervised Segmentation for Disease Localization in Chest X-Ray Images](https://mlsanity.com/paper/2007.00748/) (2020)
- [SDFN: Segmentation-based Deep Fusion Network for Thoracic Disease Classification in Chest X-ray Images](https://mlsanity.com/paper/1810.12959/) (2018)
- [Deep multiscale convolutional feature learning for weakly supervised localization of chest pathologies in X-ray images](https://mlsanity.com/paper/1808.08280/) (2018)
- [SwinCheX: Multi-label classification on chest X-ray images with transformers](https://mlsanity.com/paper/2206.04246/) (2022)
- [Radiomics-Guided Global-Local Transformer for Weakly Supervised Pathology Localization in Chest X-Rays](https://mlsanity.com/paper/2207.04394/) (2022)

## Built on by (most cited first, 1 of 1)

- [TRCGL-Net: A Long-Tailed Multi-Label Chest X-Ray Classification Framework with Generative Data Augmentation and Label Co-Occurrence Modeling](https://mlsanity.com/paper/2607.00975/) (2026)

## Abstract (arXiv)

The interpretation of chest X-rays (CXRs) poses significant challenges, particularly in achieving accurate multi-label pathology classification and spatial localization. These tasks demand different levels of annotation granularity but are frequently constrained by the scarcity of region-level (dense) annotations. We introduce CLARiTy (Class Localizing and Attention Refining Image Transformer), a vision transformer-based model for joint multi-label classification and weakly-supervised localization of thoracic pathologies. CLARiTy employs multiple class-specific tokens to generate discriminative attention maps, and a SegmentCAM module for foreground segmentation and background suppression using explicit anatomical priors. Trained on image-level labels from the NIH ChestX-ray14 dataset, it leverages distillation from a ConvNeXtV2 teacher for efficiency. Evaluated on the official NIH split, the CLARiTy-S-16-512 (a configuration of CLARiTy), achieves competitive classification performance across 14 pathologies, and state-of-the-art weakly-supervised localization performance on 8 pathologies, outperforming prior methods by 50.7%. In particular, pronounced gains occur for small pathologies like nodules and masses. The lower-resolution variant of CLARiTy, CLARiTy-S-16-224, offers high efficiency while decisively surpassing baselines, thereby having the potential for use in low-resource settings. An ablation study confirms contributions of SegmentCAM, DINO pretraining, orthogonal class token loss, and attention pooling. CLARiTy advances beyond CNN-ViT hybrids by harnessing ViT self-attention for global context and class-specific localization, refined through convolutional background suppression for precise, noise-reduced heatmaps.