mlsanityData to 7 Oct 2026

Tensor Networks Meet Neural Networks: A Survey and Future Perspectives

arXiv 2302.09019 ↗First posted 22 Jan 2023cs.LGcited by 29

Not on the map. Why?

Abstract (arXiv)

Tensor networks (TNs) and neural networks (NNs) are two fundamental data modeling approaches. TNs were introduced to solve the curse of dimensionality in large-scale tensors by converting an exponential number of dimensions to polynomial complexity. As a result, they have attracted significant attention in the fields of quantum physics and machine learning. Meanwhile, NNs have displayed exceptional performance in various applications, e.g., computer vision, natural language processing, and robotics research. Interestingly, although these two types of networks originate from different observations, they are inherently linked through the typical multilinearity structure underlying both TNs and NNs, thereby motivating a significant number of developments regarding combinations of TNs and NNs. In this paper, we refer to these combinations as tensorial neural networks~(TNNs) and present an introduction to TNNs from both data processing and model architecture perspectives. From the data perspective, we explore the capabilities of TNNs in multi-source fusion, multimodal pooling, data compression, multi-task training, and quantum data processing. From the model perspective, we examine TNNs' integration with various architectures, including Convolutional Neural Networks, Recurrent Neural Networks, Graph Neural Networks, Transformers, Large Language Models, and Quantum Neural Networks. Furthermore, this survey also explores methods for improving TNNs, examines flexible toolboxes for implementing TNNs, and documents TNN development while highlighting potential future directions. To the best of our knowledge, this is the first comprehensive survey that bridges the connections among NNs and TNs. We provide a curated list of TNNs at https://github.com/tnbar/awesome-tensorial-neural-networks.

Key facts. Cited by 29 papers in the map (data through 7 Oct 2026). Builds on 105 of its 290 references that are in the map. First posted on arXiv 22 Jan 2023.

Builds on

105 of 290 references.

References · 105

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  6. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  7. 2016Densely Connected Convolutional Networkscited by 6,691
  8. 2022Generative Adversarial Networkscited by 5,603
  9. 2014Going Deeper with Convolutionscited by 4,231
  10. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,590
  11. 2015VQA: Visual Question Answeringcited by 2,489
  12. 2013Visualizing and Understanding Convolutional Networkscited by 2,278
38 more
  1. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
  2. 2018Graph Neural Networks: A Review of Methods and Applicationscited by 1,571
  3. 2019TinyBERT: Distilling BERT for Natural Language Understandingcited by 1,153
  4. 2014Deep Learning in Neural Networks: An Overviewcited by 765
  5. 2019Importance Estimation for Neural Network Pruningcited by 531
  6. 2017Tensor Fusion Network for Multimodal Sentiment Analysiscited by 429
  7. 2020The power of quantum neural networkscited by 377
  8. 2019Parameterized quantum circuits as machine learning modelscited by 330
  9. 2014Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluationcited by 304
  10. 2018Efficient Low-rank Multimodal Fusion with Modality-Specific Factorscited by 268
  11. 2015Tensorizing Neural Networkscited by 246
  12. 2016Multimodal Compact Bilinear Pooling for Visual Question Answering and Visual Groundingcited by 243
  13. 2017Gate-Variants of Gated Recurrent Unit (GRU) Neural Networkscited by 239
  14. 2016Tensor Decomposition for Signal Processing and Machine Learningcited by 223
  15. 2014Speeding-up Convolutional Neural Networks Using Fine-tuned CP-Decompositioncited by 199
  16. 2015MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systemscited by 195
  17. 2022Language model compression with weighted low-rank factorizationcited by 192
  18. 2015Compression of Deep Convolutional Neural Networks for Fast and Low Power Mobile Applicationscited by 161
  19. 2019Quantization Networkscited by 151
  20. 2017MUTAN: Multimodal Tucker Fusion for Visual Question Answeringcited by 137
  21. 2022FacT: Factor-Tuning for Lightweight Adaptation on Vision Transformercited by 133
  22. 2016Tensor Ring Decompositioncited by 125
  23. 2016TensorLy: Tensor Learning in Pythoncited by 104
  24. 2015On the Expressive Power of Deep Learning: A Tensor Analysiscited by 97
  25. 2017Unsupervised Generative Modeling Using Matrix Product Statescited by 87
  26. 2019A Tensorized Transformer for Language Modelingcited by 85
  27. 2018Towards Quantum Machine Learning with Tensor Networkscited by 74
  28. 2020Stable Low-rank Tensor Decomposition for Compression of Convolutional Neural Networkcited by 74
  29. 2016Deep Multi-task Representation Learning: A Tensor Factorisation Approachcited by 69
  30. 2014Bayesian CP Factorization of Incomplete Tensors with Automatic Rank Determinationcited by 65
  31. 2019Tree Tensor Networks for Generative Modelingcited by 63
  32. 2020Deep Polynomial Neural Networkscited by 63
  33. 2017Tensor-Train Recurrent Neural Networks for Video Classificationcited by 59
  34. 2016Ultimate tensorization: compressing convolutional and FC layers alikecited by 55
  35. 2019Expressive power of tensor-network factorizations for probabilistic modeling, with applications from hidden Markov models to quantum machine learningcited by 55
  36. 2020Convolutional Tensor-Train LSTM for Spatio-temporal Learningcited by 54
  37. 2017Tensor Networks for Dimensionality Reduction and Large-Scale Optimizations. Part 2 Applications and Future Perspectivescited by 53
  38. 2021Tensor Methods in Computer Vision and Deep Learningcited by 52

50 most cited of 105 All 105 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 5; 2024: 8; 2025: 10; 2026: 6.

29 papers.

Sort:

  1. 2024CompactifAI: Extreme Compression of Large Language Models using Quantum-Inspired Tensor Networkscited by 29
  2. 2023Tensor networks for interpretable and efficient quantum-inspired machine learningcited by 16
  3. 2023Reusing Pretrained Models by Multi-linear Operators for Efficient Trainingcited by 15
  4. 2023Quantum-Inspired Machine Learning: a Surveycited by 11
  5. 2026KromHC: Manifold-Constrained Hyper-Connections with Kronecker-Product Residual Matricescited by 10
  6. 2023On the Opportunities of Green Computing: A Surveycited by 7
  7. 2024Quantum Large Language Models via Tensor Network Disentanglerscited by 7
  8. 2025F-INR: Functional Tensor Decomposition for Implicit Neural Representationscited by 6
  9. 2025IDInit: A Universal and Stable Initialization Method for Neural Network Trainingcited by 5
  10. 2025Tensor Network for Anomaly Detection in the Latent Space of Proton Collision Events at the LHCcited by 5
  11. 2024Ultra-low latency quantum-inspired machine learning predictors implemented on FPGAcited by 4
  12. 2025tn4ml: Tensor Network Training and Customization for Machine Learningcited by 4
17 more
  1. 2024Quantum Theory and Application of Contextual Optimal Transportcited by 3
  2. 2024Cross Space and Time: A Spatio-Temporal Unitized Model for Traffic Flow Forecastingcited by 3
  3. 2025Tensorization is a powerful but underexplored tool for compression and interpretability of neural networkscited by 3
  4. 2025MetaLoRA: Tensor-Enhanced Adaptive Low-Rank Fine-tuningcited by 2
  5. 2023Advocating for the Silent: Enhancing Federated Generalization for Non-Participating Clientscited by 1
  6. 2024T'ecnicas Quantum-Inspired en Tensor Networks para Contextos Industrialescited by 0
  7. 2024Subspace Node Pruningcited by 0
  8. 2024No-Free-Lunch Theories for Tensor-Network Machine Learning Modelscited by 0
  9. 2025Constructing 3D Rotational Invariance and Equivariance with Symmetric Tensor Networkscited by 0
  10. 2025Data Understanding Survey: Pursuing Improved Dataset Characterization Via Tensor-based Methodscited by 0
  11. 2025KARIPAP: Quantum-Inspired Tensor Network Compression of Large Language Models Using Infinite Projected Entangled Pair States and Tensor Renormalization Groupcited by 0
  12. 2025Tensor Computing Interface: An Application-Oriented, Lightweight Interface for Portable High-Performance Tensor Network Applicationscited by 0
  13. 2026Tensor-Augmented Convolutional Neural Networks: Enhancing Expressivity with Generic Tensor Kernelscited by 0
  14. 2026Quantum-inspired tensor networks in machine learning modelscited by 0
  15. 2026Entanglement is Half the Story: Post-Selection vs. Partial Tracescited by 0
  16. 2026T-GINEE: A Tensor-Based Multilayer Graph Representation Learningcited by 0
  17. 2026Automatically Differentiable Nonlinear Tensor Networks (ADNTNs) for Exponential Parameter Compression of Deep Neural Networkscited by 0