mlsanityData to 7 Oct 2026

VaultGemma: A Differentially Private Gemma Model

arXiv 2510.15001 ↗First posted 15 Oct 2025cs.CR cs.AIcited by 14

Not on the map. Why?

Abstract (arXiv)

We introduce VaultGemma 1B, a 1 billion parameter model within the Gemma family, fully trained with differential privacy. Pretrained on the identical data mixture used for the Gemma 2 series, VaultGemma 1B represents a significant step forward in privacy-preserving large language models. We openly release this model to the community

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 15 of its 23 references that are in the map. First posted on arXiv 15 Oct 2025.

Builds on

15 of 23 references.

References · 15

  1. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
  2. 2022Training Compute-Optimal Large Language Modelscited by 2,927
  3. 2016Deep Learning with Differential Privacycited by 2,682
  4. 2020Extracting Training Data from Large Language Modelscited by 2,144
  5. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
  6. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,928
  7. 2019Root Mean Square Layer Normalizationcited by 1,425
  8. 2023Analyzing Leakage of Personally Identifiable Information in Language Modelscited by 278
  9. 2021Practical and Private (Deep) Learning without Sampling or Shufflingcited by 169
  10. 2023Emergent and Predictable Memorization in Large Language Modelscited by 147
  11. 2022Position: Considerations for Differentially Private Learning with Large-Scale Public Pretrainingcited by 86
  12. 2022Connect the Dots: Tighter Discrete Approximations of Privacy Loss Distributionscited by 50
3 more
  1. 2024Recite, Reconstruct, Recollect: Memorization in LMs as a Multifaceted Phenomenoncited by 30
  2. 2025Scaling Laws for Differentially Private Language Modelscited by 22
  3. 2024Scalable DP-SGD: Shuffling vs. Poisson Subsamplingcited by 16

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 13.

14 papers.

Sort:

By area:

  1. 2025ContextLeak: Auditing Leakage in Private In-Context Learning Methodscited by 4
  2. 2026Efficient privacy loss accounting for subsampling and random allocationcited by 3
  3. 2026Differentially Private Language Generation and Identification in the Limitcited by 3
  4. 2026JAX-Privacy: A library for differentially private machine learningcited by 2
  5. 2026Your Privacy Depends on Others: Collusion Vulnerabilities in Individual Differential Privacycited by 1
  6. 2026Decomposing Memorization Reduction in Privacy-Preserving Fine-Tuning of SLMs for CSIRTscited by 1
  7. 2026Demystifying the Privacy-Utility Trade-off in LLM Interactionscited by 1
  8. 2026Private PoEtry: Private In-Context Learning via Product of Expertscited by 0
  9. 2026MAPLE: Metadata Augmented Private Language Evolutioncited by 0
  10. 2026How Does Differential Privacy Affect Social Bias in LLMs? A Systematic Evaluationcited by 0
  11. 2026From Privacy to Generalization: Linear Max-Information Bounds for Differentially Private Learning Algorithmscited by 0
  12. 2026Dithered Gaussian Mechanism for Randomness-Efficient Differential Privacycited by 0
2 more
  1. 2026Private Direct Preference Optimization for LLM Alignmentcited by 0
  2. 2026The Privacy-Hallucination Tradeoff in Differentially Private Language Modelscited by 0