mlsanityData to 14 Oct 2026

Modeling Neural Networks with Privacy Using Neural Stochastic Differential Equations

arXiv 2501.06686 ↗First posted 12 Jan 2025cs.CR cs.LGcited by 0

On the map: LLM safety and security → Membership inference attacks

Abstract (arXiv)

In this work, we study the feasibility of using neural ordinary differential equations (NODEs) to model systems with intrinsic privacy properties. Unlike conventional feedforward neural networks, which have unlimited expressivity and can represent arbitrary mappings between inputs and outputs, NODEs constrain their learning to the solution of a system of differential equations. We first examine whether this constraint reduces memorization and, consequently, the membership inference risks associated with NODEs. We conduct a comprehensive evaluation of NODEs under membership inference attacks and show that they exhibit twice the resistance compared to conventional models such as ResNets. By analyzing the variance in membership risks across different NODE models, we find that their limited expressivity leads to reduced overfitting to the training data. We then demonstrate, both theoretically and empirically, that membership inference risks can be further mitigated by utilizing a stochastic variant of NODEs: neural stochastic differential equations (NSDEs). We show that NSDEs are differentially-private (DP) learners that provide the same provable privacy guarantees as DPSGD, the de-facto mechanism for training private models. NSDEs are also effective in mitigating membership inference attacks, achieving risk levels comparable to private models trained with DP-SGD while offering an improved privacyutility trade-off. Moreover, we propose a drop-in-replacement strategy that efficiently integrates NSDEs into conventional feedforward architectures to enhance their privacy.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Membership inference attacks, one of 12 directions in LLM safety and security. Builds on 47 of its 82 references that are in the map. First posted on arXiv 12 Jan 2025.

Builds on

47 of 82 references.

Closest ancestors · 3

Same direction, newest first

  1. 2023Low-Cost High-Power Membership Inference Attacks6 Dec
  2. 2021Membership Inference Attacks From First Principles7 Dec
  3. 2021On the Importance of Difficulty Calibration in Membership Inference Attacks15 Nov

Other references · 44

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  3. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  4. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
  5. 2018Neural Ordinary Differential Equationscited by 3,779
  6. 2016Deep Learning with Differential Privacycited by 2,728
  7. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,281
  8. 2020Extracting Training Data from Large Language Modelscited by 2,189
  9. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 864
  10. 2023Extracting Training Data from Diffusion Modelscited by 732
  11. 2019Hamiltonian Neural Networkscited by 712
  12. 2020Neural Controlled Differential Equations for Irregular Time Seriescited by 506
32 more
  1. 2020What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimationcited by 497
  2. 2019Augmented Neural ODEscited by 397
  3. 2020Label-Only Membership Inference Attackscited by 388
  4. 2020Systematic Evaluation of Privacy Risks of Machine Learning Modelscited by 292
  5. 2020Scalable Gradients for Stochastic Differential Equationscited by 271
  6. 2020Differentially Private Learning Needs Better Features (or Much More Data)cited by 235
  7. 2020How to train your neural ODE: the world of Jacobian and kinetic regularizationcited by 228
  8. 2019MemGuard: Defending against Black-Box Membership Inference Attacks via Adversarial Examplescited by 197
  9. 2021Neural SDEs as Infinite-Dimensional GANscited by 161
  10. 2019Neural SDE: Stabilizing Neural ODE Networks with Stochastic Noisecited by 101
  11. 2021Stable Neural ODE with Lyapunov-Stable Equilibrium Points for Defending Against Adversarial Attackscited by 91
  12. 2019On Robustness of Neural Ordinary Differential Equationscited by 83
  13. 2024ClimODE: Climate and Weather Forecasting with Physics-informed Neural ODEscited by 82
  14. 2020Learning Differential Equations that are Easy to Solvecited by 81
  15. 2020SDE-Net: Equipping Deep Neural Networks with Uncertainty Estimatescited by 75
  16. 2020On Second Order Behaviour in Augmented Neural ODEscited by 68
  17. 2020STEER: Simple Temporal Regularization For Neural ODEscited by 58
  18. 2021Machine learning structure preserving brackets for forecasting irreversible processescited by 52
  19. 2023Direct Parameterization of Lipschitz-Bounded Deep Networkscited by 51
  20. 2022GT-GAN: General Purpose Time Series Synthesis with Generative Adversarial Networkscited by 44
  21. 2022Are Your Sensitive Attributes Private? Novel Model Inversion Attribute Inference Attacks on Classification Modelscited by 43
  22. 2021Heavy Ball Neural Ordinary Differential Equationscited by 42
  23. 2016You are Who You Know and How You Behave: Attribute Inference Attacks via Users' Social Friends and Behaviorscited by 34
  24. 2020Joint Item Recommendation and Attribute Inference: An Adaptive Graph Convolutional Network Approachcited by 30
  25. 2023Reversible and irreversible bracket-based dynamics for deep graph neural networkscited by 26
  26. 2020On the Effectiveness of Regularization Against Membership Inference Attackscited by 24
  27. 2020Adversarial Robustness of Stabilized NeuralODEs Might be from Obfuscated Gradientscited by 22
  28. 2020Stochasticity in Neural ODEs: An Empirical Studycited by 13
  29. 2022How Does a Deep Learning Model Architecture Impact Its Privacy? A Comprehensive Study of Privacy Attacks on CNNs and Transformerscited by 8
  30. 2021Improving Robustness and Uncertainty Modelling in Neural Ordinary Differential Equationscited by 7
  31. 2022AdamNODEs: When Neural ODE Meets Adaptive Moment Estimationcited by 3
  32. 2023Efficient Certified Training and Robustness Verification of Neural ODEscited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2021On the Privacy Risks of Deploying Recurrent Neural Networks in Machine Learning Models
  2. 2023Preserving Node-level Privacy in Graph Neural Networks
  3. 2025On the Privacy Risks of Spiking Neural Networks: A Membership Inference Analysis
  4. 2025On the Privacy-Preserving Properties of Spiking Neural Networks with Unique Surrogate Gradients and Quantization Levels
  5. 2022Additive Logistic Mechanism for Privacy-Preserving Self-Supervised Learning
  6. 2021Generalization Techniques Empirically Outperform Differential Privacy against Membership Inference
  7. 2024Membership Privacy Evaluation in Deep Spiking Neural Networks
  8. 2026Bayesian Membership Privacy for Graph Neural Networks