Tensorization of neural networks for improved privacy and interpretability
Not on the map. Why?
Abstract (arXiv)
We present a tensorization algorithm for constructing tensor train/matrix product state (MPS) representations of functions, drawing on sketching and cross interpolation ideas. The method only requires black-box access to the target function and a small set of sample points defining the domain of interest. Thus, it is particularly well-suited for machine learning models, where the domain of interest is naturally defined by the training dataset. We show that this approach can be used to enhance the privacy and interpretability of neural network models. Specifically, we apply our decomposition to (i) obfuscate neural networks whose parameters encode patterns tied to the training data distribution, and (ii) estimate topological phases of matter that are easily accessible from the MPS representation. Additionally, we show that this tensorization can serve as an efficient initialization method for optimizing MPS in general settings, and that, for model compression, our algorithm achieves a superior trade-off between memory and time complexity compared to conventional tensorization methods of neural networks.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 50 of its 126 references that are in the map. First posted on arXiv 10 Jan 2025.
Builds on
50 of 126 references.
References · 50
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
- 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
- 2021Zero-Shot Text-to-Image Generationcited by 4,276
- 2014Going Deeper with Convolutionscited by 4,231
- 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,590
- 2016Deep Learning with Differential Privacycited by 2,682
- 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,358
- 2018A Survey Of Methods For Explaining Black Box Modelscited by 970
- 2018A Convergence Theory for Deep Learning via Over-Parameterizationcited by 784
38 more
- 2018This Looks Like That: Deep Learning for Interpretable Image Recognitioncited by 731
- 2023Extracting Training Data from Diffusion Modelscited by 723
- 2016Stealing Machine Learning Models via Prediction APIscited by 602
- 2023Scalable Extraction of Training Data from (Production) Language Modelscited by 415
- 2019Differential Privacy Has Disparate Impact on Model Accuracycited by 316
- 2020A Survey on Neural Network Interpretabilitycited by 291
- 2014Sketching as a Tool for Numerical Linear Algebracited by 285
- 2015Tensorizing Neural Networkscited by 246
- 2020Predicting Many Properties of a Quantum System from Very Few Measurementscited by 229
- 2020Auditing Differentially Private Machine Learning: How Private is Private SGD?cited by 213
- 2014Speeding-up Convolutional Neural Networks Using Fine-tuned CP-Decompositioncited by 199
- 2021Large Scale Private Learning via Low-rank Reparametrizationcited by 87
- 2019A Tensorized Transformer for Language Modelingcited by 85
- 2019Tree Tensor Networks for Generative Modelingcited by 63
- 2019Local Differential Privacy for Deep Learningcited by 46
- 2017Machine Learning by Unitary Tensor Network of Hierarchical Tree Structurecited by 39
- 2020Anomaly Detection with Tensor Networkscited by 26
- 2020Towards Compact Neural Networks via End-to-End Training: A Bayesian Tensor Approach with Automatic Rank Determinationcited by 17
- 2022Generative modeling with projected entangled-pair statescited by 14
- 2023Tensor Networks for Explainable Machine Learning in Cybersecuritycited by 14
- 2022Generative modeling via tensor train sketchingcited by 13
- 2020Privacy-preserving Learning via Deep Net Pruningcited by 12
- 2022Privacy-preserving machine learning with tensor networkscited by 8
- 2023Generative Modeling via Hierarchical Tensor Sketchingcited by 8
- 2023Efficient GPT Model Pre-training using Tensor Train Matrix Representationcited by 8
- 2021Tensor Random Projection for Low Memory Dimension Reductioncited by 7
- 2021Explainable Natural Language Processing with Matrix Product Statescited by 7
- 2022tntorch: Tensor Network Learning with PyTorchcited by 7
- 2024Quantum Large Language Models via Tensor Network Disentanglerscited by 7
- 2022Improvements to Gradient Descent Methods for Quantum Tensor Network Machine Learningcited by 6
- 2023Investigating Topological Order using Recurrent Neural Networkscited by 6
- 2020Defence against adversarial attacks using classical and quantum-enhanced Boltzmann machinescited by 5
- 2022Generative Modeling via Tree Tensor Network Statescited by 5
- 2023Differential Privacy Meets Neural Network Pruningcited by 5
- 2023TensorKrowch: Smooth integration of tensor networks in machine learningcited by 5
- 2024A Matrix Product State Model for Simultaneous Classification and Generationcited by 5
- 2022High-dimensional density estimation with tensorizing flowcited by 4
- 2024Tensor network compressibility of convolutional modelscited by 2
Built on by
Citing papers per year: 2025: 1; 2026: 2.
3 papers.