mlsanityData to 14 Oct 2026

A Note On Interpreting Canary Exposure

arXiv 2306.00133 ↗First posted 31 May 2023cs.CR cs.LGcited by 6

Not on the map. Why?

Abstract (arXiv)

Canary exposure, introduced in Carlini et al. is frequently used to empirically evaluate, or audit, the privacy of machine learning model training. The goal of this note is to provide some intuition on how to interpret canary exposure, including by relating it to membership inference attacks and differential privacy.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 20 of its 22 references that are in the map. First posted on arXiv 31 May 2023.

Builds on

20 of 22 references.

References · 20

  1. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,281
  2. 2023PaLM 2 Technical Reportcited by 1,162
  3. 2021Membership Inference Attacks From First Principlescited by 878
  4. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 864
  5. 2013The Composition Theorem for Differential Privacycited by 297
  6. 2020Auditing Differentially Private Machine Learning: How Private is Private SGD?cited by 218
  7. 2022Reconstructing Training Data with Informed Adversariescited by 148
  8. 2021On the Importance of Difficulty Calibration in Membership Inference Attackscited by 147
  9. 2023Privacy Auditing with One (1) Training Runcited by 123
  10. 2022Truth Serum: Poisoning Machine Learning Models to Reveal Their Secretscited by 101
  11. 2019Analyzing Information Leakage of Updates to Natural Language Modelscited by 90
  12. 2020Training Production Language Models without Memorizing User Datacited by 68
8 more
  1. 2021Antipodes of Label Differential Privacy: PATE and ALIBIcited by 52
  2. 2020Understanding Unintended Memorization in Federated Learningcited by 36
  3. 2022Bayesian Estimation of Differential Privacycited by 33
  4. 2022Defending against Reconstruction Attacks with R'enyi Differential Privacycited by 32
  5. 2022A General Framework for Auditing Differentially Private Machine Learningcited by 25
  6. 2023Unleashing the Power of Randomization in Auditing Differentially Private MLcited by 20
  7. 2021Wide Network Learning with Differential Privacycited by 17
  8. 2022Detecting Unintended Memorization in Language-Model-Fused ASRcited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 4; 2025: 1; 2026: 1.

6 papers.

  1. 2024Privacy-Preserving Instructions for Aligning Large Language Modelscited by 35
  2. 2024The Responsible Foundation Model Development Cheatsheet: A Review of Tools & Resourcescited by 11
  3. 2024Auditing Differential Privacy Guarantees Using Density Estimationcited by 6
  4. 2024Unlearning in- vs. out-of-distribution data in LLMs under gradient-based methodcited by 4
  5. 2026Benchmarking Empirical Privacy Protection for Adaptations of Large Language Modelscited by 4
  6. 2025Covert Attacks on Machine Learning Training in Passively Secure MPCcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2026Explanations Leak: Membership Inference with Differential Privacy and Active Learning Defense
  2. 2024Explaining the Model, Protecting Your Data: Revealing and Mitigating the Data Privacy Risks of Post-Hoc Model Explanations via Membership Inference
  3. 2025Membership Inference Attacks Beyond Overfitting
  4. 2024Free Record-Level Privacy Risk Evaluation Through Artifact-Based Methods
  5. 2023A Cautionary Tale: On the Role of Reference Data in Empirical Privacy Defenses
  6. 2025A hierarchical approach for assessing the vulnerability of tree-based classification models to membership inference attack
  7. 2026From Multiplicity to Vulnerability: Privacy Amplification Risk from One-Dataset-Multiple-Model Exposure ↗
  8. 2026Detectability in Diversity: Improved Canary Crafting for Privacy Auditing in One Run