# Dataset Inference for Self-Supervised Models

arXiv: https://arxiv.org/abs/2209.09024 · HTML: https://mlsanity.com/paper/2209.09024/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 53 references that are in the map.
- First posted on arXiv 16 Sep 2022.

## Builds on (most cited first, 30 of 32)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Exploring Simple Siamese Representation Learning](https://mlsanity.com/paper/2011.10566/) (2020)
- [BioBERT: a pre-trained biomedical language representation model for biomedical text mining](https://mlsanity.com/paper/1901.08746/) (2019)
- [Similarity of Neural Network Representations Revisited](https://mlsanity.com/paper/1905.00414/) (2019)
- [Learning Representations by Maximizing Mutual Information Across Views](https://mlsanity.com/paper/1906.00910/) (2019)
- [Stealing Machine Learning Models via Prediction APIs](https://mlsanity.com/paper/1609.02943/) (2016)
- [A Survey on Contrastive Self-supervised Learning](https://mlsanity.com/paper/2011.00362/) (2020)
- [Insights on representational similarity in neural networks with canonical correlation](https://mlsanity.com/paper/1806.05759/) (2018)
- [Provable Guarantees for Self-Supervised Deep Learning with Spectral Contrastive Loss](https://mlsanity.com/paper/2106.04156/) (2021)
- [Failing Loudly: An Empirical Study of Methods for Detecting Dataset Shift](https://mlsanity.com/paper/1810.11953/) (2018)
- [High Accuracy and High Fidelity Extraction of Neural Networks](https://mlsanity.com/paper/1909.01838/) (2019)
- [Formal Limitations on the Measurement of Mutual Information](https://mlsanity.com/paper/1811.04251/) (2018)
- [PRADA: Protecting against DNN Model Stealing Attacks](https://mlsanity.com/paper/1805.02628/) (2018)
- [Entangled Watermarks as a Defense against Model Extraction](https://mlsanity.com/paper/2002.12200/) (2020)
- [Dataset Inference: Ownership Resolution in Machine Learning](https://mlsanity.com/paper/2104.10706/) (2021)
- [The Cost of Training NLP Models: A Concise Overview](https://mlsanity.com/paper/2004.08900/) (2020)
- [Proof-of-Learning: Definitions and Practice](https://mlsanity.com/paper/2103.05633/) (2021)
- [Prediction Poisoning: Towards Defenses Against DNN Model Stealing Attacks](https://mlsanity.com/paper/1906.10908/) (2019)
- [EncoderMI: Membership Inference against Pre-trained Encoders in Contrastive Learning](https://mlsanity.com/paper/2108.11023/) (2021)
- [A Systematic Review on Model Watermarking for Neural Networks](https://mlsanity.com/paper/2009.12153/) (2020)
- [Non-Transferable Learning: A New Approach for Model Ownership Verification and Applicability Authorization](https://mlsanity.com/paper/2106.06916/) (2021)
- [SSLGuard: A Watermarking Scheme for Self-supervised Learning Pre-trained Encoders](https://mlsanity.com/paper/2201.11692/) (2022)
- [REFIT: A Unified Watermark Removal Framework For Deep Learning Systems With Limited Data](https://mlsanity.com/paper/1911.07205/) (2019)
- [Contrasting Contrastive Self-Supervised Representation Learning Pipelines](https://mlsanity.com/paper/2103.14005/) (2021)
- [Quantifying and Mitigating Privacy Risks of Contrastive Learning](https://mlsanity.com/paper/2102.04140/) (2021)
- [Concept Generalization in Visual Representation Learning](https://mlsanity.com/paper/2012.05649/) (2020)
- [Can't Steal? Cont-Steal! Contrastive Stealing Attacks Against Image Encoders](https://mlsanity.com/paper/2201.07513/) (2022)

## Built on by (most cited first, 30 of 36)

- [LLM Dataset Inference: Did you train on my dataset?](https://mlsanity.com/paper/2406.06443/) (2024)
- [Explanation as a Watermark: Towards Harmless and Multi-bit Model Ownership Verification via Watermarking Feature Attribution](https://mlsanity.com/paper/2405.04825/) (2024)
- [CDI: Copyrighted Data Identification in Diffusion Models](https://mlsanity.com/paper/2411.12858/) (2024)
- [Ward: Provable RAG Dataset Inference via LLM Watermarks](https://mlsanity.com/paper/2410.03537/) (2024)
- [SoK: Dataset Copyright Auditing in Machine Learning Systems](https://mlsanity.com/paper/2410.16618/) (2024)
- [Privacy Attacks on Image AutoRegressive Models](https://mlsanity.com/paper/2502.02514/) (2025)
- [ORL-AUDITOR: Dataset Auditing in Offline Deep Reinforcement Learning](https://mlsanity.com/paper/2309.03081/) (2023)
- [A Unified Membership Inference Method for Visual Self-supervised Encoder via Part-aware Capability](https://mlsanity.com/paper/2404.02462/) (2024)
- [Bucks for Buckets (B4B): Active Defenses Against Stealing Encoders](https://mlsanity.com/paper/2310.08571/) (2023)
- [TUNI: A Textual Unimodal Detector for Identity Inference in CLIP Models](https://mlsanity.com/paper/2405.14517/) (2024)
- [DATABench: Evaluating Dataset Auditing in Deep Learning from an Adversarial Perspective](https://mlsanity.com/paper/2507.05622/) (2025)
- [Unlocking Post-hoc Dataset Inference with Synthetic Data](https://mlsanity.com/paper/2506.15271/) (2025)
- [Are You Copying My Prompt? Protecting the Copyright of Vision Prompt for VPaaS via Watermark](https://mlsanity.com/paper/2405.15161/) (2024)
- [FDINet: Protecting against DNN Model Extraction via Feature Distortion Index](https://mlsanity.com/paper/2306.11338/) (2023)
- [Hey, That's My Data! Token-Only Dataset Inference in Large Language Models](https://mlsanity.com/paper/2506.06057/) (2025)
- [SWAP: Towards Copyright Auditing of Soft Prompts via Sequential Watermarking](https://mlsanity.com/paper/2511.04711/) (2025)
- [Refinement Provenance Inference: Detecting LLM-Refined Training Prompts from Model Behavior](https://mlsanity.com/paper/2601.01966/) (2026)
- [ADAGE: Active Defenses Against GNN Extraction](https://mlsanity.com/paper/2503.00065/) (2025)
- [Dataset Ownership Verification for Pre-trained Masked Models](https://mlsanity.com/paper/2507.12022/) (2025)
- [Evading Data Provenance in Deep Neural Networks](https://mlsanity.com/paper/2508.01074/) (2025)
- [Membership and Dataset Inference Attacks on Large Audio Generative Models](https://mlsanity.com/paper/2512.09654/) (2025)
- [Ownership Protection of Generative Adversarial Networks](https://mlsanity.com/paper/2306.05233/) (2023)
- [SSL-Auth: An Authentication Framework by Fragile Watermarking for Pre-trained Encoders in Self-supervised Learning](https://mlsanity.com/paper/2308.04673/) (2023)
- [PATFinger: Prompt-Adapted Transferable Fingerprinting against Unauthorized Multimodal Dataset Usage](https://mlsanity.com/paper/2504.11509/) (2025)
- [A Unified and Scalable Membership Inference Method for Visual Self-supervised Encoder via Part-aware Capability](https://mlsanity.com/paper/2505.10351/) (2025)
- [Efficient and Privacy-Preserving Soft Prompt Transfer for LLMs](https://mlsanity.com/paper/2506.16196/) (2025)
- [VICTOR: Dataset Copyright Auditing in Video Recognition Systems](https://mlsanity.com/paper/2512.14439/) (2025)
- [Unraveling Attacks in Machine Learning-based IoT Ecosystems: A Survey and the Open Libraries Behind Them](https://mlsanity.com/paper/2401.11723/) (2024)
- [A2-DIDM: Privacy-preserving Accumulator-enabled Auditing for Distributed Identity of DNN Model](https://mlsanity.com/paper/2405.04108/) (2024)
- [Pre-trained Encoder Inference: Revealing Upstream Encoders In Downstream Machine Learning Services](https://mlsanity.com/paper/2408.02814/) (2024)

## Abstract (arXiv)

Self-supervised models are increasingly prevalent in machine learning (ML) since they reduce the need for expensively labeled data. Because of their versatility in downstream applications, they are increasingly used as a service exposed via public APIs. At the same time, these encoder models are particularly vulnerable to model stealing attacks due to the high dimensionality of vector representations they output. Yet, encoders remain undefended: existing mitigation strategies for stealing attacks focus on supervised learning. We introduce a new dataset inference defense, which uses the private training set of the victim encoder model to attribute its ownership in the event of stealing. The intuition is that the log-likelihood of an encoder's output representations is higher on the victim's training data than on test data if it is stolen from the victim, but not if it is independently trained. We compute this log-likelihood using density estimation models. As part of our evaluation, we also propose measuring the fidelity of stolen encoders and quantifying the effectiveness of the theft detection without involving downstream tasks; instead, we leverage mutual information and distance measurements. Our extensive empirical results in the vision domain demonstrate that dataset inference is a promising direction for defending self-supervised models against model stealing.