mlsanityData to 7 Oct 2026

NeuroMixGDP: A Neural Collapse-Inspired Random Mixup for Private Data Release

arXiv 2202.06467 ↗First posted 14 Feb 2022cs.LG cs.CVcited by 2

Not on the map. Why?

Abstract (arXiv)

Privacy-preserving data release algorithms have gained increasing attention for their ability to protect user privacy while enabling downstream machine learning tasks. However, the utility of current popular algorithms is not always satisfactory. Mixup of raw data provides a new way of data augmentation, which can help improve utility. However, its performance drastically deteriorates when differential privacy (DP) noise is added. To address this issue, this paper draws inspiration from the recently observed Neural Collapse (NC) phenomenon, which states that the last layer features of a neural network concentrate on the vertices of a simplex as Equiangular Tight Frame (ETF). We propose a scheme to mixup the Neural Collapse features to exploit the ETF simplex structure and release noisy mixed features to enhance the utility of the released data. By using Gaussian Differential Privacy (GDP), we obtain an asymptotic rate for the optimal mixup degree. To further enhance the utility and address the label collapse issue when the mixup degree is large, we propose a Hierarchical sampling method to stratify the mixup samples on a small number of classes. This method remarkably improves utility when the number of classes is large. Extensive experiments demonstrate the effectiveness of our proposed method in protecting against attacks and improving utility. In particular, our approach shows significantly improved utility compared to directly training classification networks with DPSGD on CIFAR100 and MiniImagenet datasets, highlighting the benefits of using privacy-preserving data release. We release reproducible code in https://github.com/Lidonghao1996/NeuroMixGDP.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 31 of its 57 references that are in the map. First posted on arXiv 14 Feb 2022.

Builds on

31 of 57 references.

References · 31

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  3. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  4. 2016Deep Learning with Differential Privacycited by 2,682
  5. 2018Group Normalizationcited by 1,514
  6. 2020Big Self-Supervised Models are Strong Semi-Supervised Learnerscited by 1,375
  7. 2020Prevalence of Neural Collapse during the terminal phase of deep learning trainingcited by 713
  8. 2017Renyi Differential Privacycited by 643
  9. 2018ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Modelscited by 475
  10. 2018Differentially Private Generative Adversarial Networkcited by 235
  11. 2018Subsampled R'enyi Differential Privacy and Analytical Moments Accountantcited by 234
  12. 2020Differentially Private Learning Needs Better Features (or Much More Data)cited by 233
19 more
  1. 2021A Geometric Analysis of Neural Collapse with Unconstrained Featurescited by 226
  2. 2021Exploring Deep Neural Networks via Layer-Peeled Model: Minority Collapse in Imbalanced Trainingcited by 166
  3. 2021Neural Collapse Under MSE Loss: Proximity to and Dynamics on the Central Pathcited by 156
  4. 2020Neural collapse with unconstrained featurescited by 140
  5. 2020Tempered Sigmoid Activations for Deep Learning with Differential Privacycited by 121
  6. 2020GS-WGAN: A Gradient-Sanitized Approach for Learning Differentially Private Generatorscited by 117
  7. 2019Deep Learning with Gaussian Differential Privacycited by 112
  8. 2021Winning the NIST Contest: A scalable and general approach to differentially private synthetic datacited by 101
  9. 2021On the Role of Neural Collapse in Transfer Learningcited by 100
  10. 2021An Unconstrained Layer-Peeled Perspective on Neural Collapsecited by 87
  11. 2020DP-MERF: Differentially Private Mean Embeddings with Random Features for Practical Privacy-Preserving Data Generationcited by 78
  12. 2021Don't Generate Me: Training Differentially Private Generative Models with Sinkhorn Divergencecited by 54
  13. 2019G-PATE: Scalable Differentially Private Data Generator via Private Aggregation of Teacher Discriminatorscited by 53
  14. 2014Deep Roto-Translation Scattering for Object Classificationcited by 49
  15. 2021DataLens: Scalable Privacy Preserving Training via Gradient Compression and Aggregationcited by 31
  16. 2021PEARL: Data Synthesis via Private Embeddings and Adversarial Reconstruction Learningcited by 23
  17. 2020P3GM: Private High-Dimensional Data Release via Privacy Preserving Phased Generative Modelcited by 21
  18. 2021Hermite Polynomial Features for Private Data Generationcited by 21
  19. 2021Towards Understanding the Data Dependency of Mixup-style Trainingcited by 19

Built on by

Cited by year

Citing papers per year: 2024: 2.

2 papers.

  1. 2024Linguistic Collapse: Neural Collapse in (Large) Language Modelscited by 36
  2. 2024Collapsed Language Models Promote Fairnesscited by 1