# ATLAS: Automatically Detecting Discrepancies Between Privacy Policies and Privacy Labels

arXiv: https://arxiv.org/abs/2306.09247 · HTML: https://mlsanity.com/paper/2306.09247/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 49 references that are in the map.
- First posted on arXiv 24 May 2023.

## Builds on (most cited first, 14 of 14)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [Convolutional Neural Networks for Sentence Classification](https://mlsanity.com/paper/1408.5882/) (2014)
- [Learning from Noisy Labels with Deep Neural Networks: A Survey](https://mlsanity.com/paper/2007.08199/) (2020)
- [DocBERT: BERT for Document Classification](https://mlsanity.com/paper/1904.08398/) (2019)
- [Question Answering for Privacy Policies: Combining Computational and Legal Perspectives](https://mlsanity.com/paper/1911.00841/) (2019)
- [Evaluating Multi-label Classifiers with Noisy Labels](https://mlsanity.com/paper/2102.08427/) (2021)
- [Goodbye Tracking? Impact of iOS App Tracking Transparency and Privacy Labels](https://mlsanity.com/paper/2204.03556/) (2022)
- [Longitudinal Analysis of Privacy Labels in the Apple App Store](https://mlsanity.com/paper/2206.02658/) (2022)
- [Lalaine: Measuring and Characterizing Non-Compliance of Apple Privacy Labels at Scale](https://mlsanity.com/paper/2206.06274/) (2022)

## Built on by (most cited first, 5 of 5)

- [PolicyLR: A Logic Representation For Privacy Policies](https://mlsanity.com/paper/2408.14830/) (2024)
- [Generating Privacy Stories From Software Documentation](https://mlsanity.com/paper/2506.23014/) (2025)
- [Enabling Multilingual Privacy Policy Audits: Large-Scale Analysis of Spanish Mobile Apps](https://mlsanity.com/paper/2607.18424/) (2026)
- [SoK: From Generation to Consumption of Privacy Documents in Software Systems](https://mlsanity.com/paper/2608.12511/) (2026)
- [Privacy Washing: Detecting Internal Contradictions in Privacy Policies](https://mlsanity.com/paper/2609.02055/) (2026)

## Abstract (arXiv)

Privacy policies are long, complex documents that end-users seldom read. Privacy labels aim to ameliorate these issues by providing succinct summaries of salient data practices. In December 2020, Apple began requiring that app developers submit privacy labels describing their apps' data practices. Yet, research suggests that app developers often struggle to do so. In this paper, we automatically identify possible discrepancies between mobile app privacy policies and their privacy labels. Such discrepancies could be indicators of potential privacy compliance issues. We introduce the Automated Privacy Label Analysis System (ATLAS). ATLAS includes three components: a pipeline to systematically retrieve iOS App Store listings and privacy policies; an ensemble-based classifier capable of predicting privacy labels from the text of privacy policies with 91.3% accuracy using state-of-the-art NLP techniques; and a discrepancy analysis mechanism that enables a large-scale privacy analysis of the iOS App Store. Our system has enabled us to analyze 354,725 iOS apps. We find several interesting trends. For example, only 40.3% of apps in the App Store provide easily accessible privacy policies, and only 29.6% of apps provide both accessible privacy policies and privacy labels. Among apps that provide both, 88.0% have at least one possible discrepancy between the text of their privacy policy and their privacy label, which could be indicative of a potential compliance issue. We find that, on average, apps have 5.32 such potential compliance issues. We hope that ATLAS will help app developers, researchers, regulators, and mobile app stores alike. For example, app developers could use our classifier to check for discrepancies between their privacy policies and privacy labels, and regulators could use our system to help review apps at scale for potential compliance issues.