mlsanityData to 14 Oct 2026

Tensor Methods in Computer Vision and Deep Learning

arXiv 2107.03436 ↗First posted 7 Jul 2021cs.CVcited by 52

Not on the map. Why?

Abstract (arXiv)

Tensors, or multidimensional arrays, are data structures that can naturally represent visual data of multiple dimensions. Inherently able to efficiently capture structured, latent semantic spaces and high-order interactions, tensors have a long history of applications in a wide span of computer vision problems. With the advent of the deep learning paradigm shift in computer vision, tensors have become even more fundamental. Indeed, essential ingredients in modern deep learning architectures, such as convolutions and attention mechanisms, can readily be considered as tensor mappings. In effect, tensor methods are increasingly finding significant applications in deep learning, including the design of memory and compute efficient network architectures, improving robustness to random noise and adversarial attacks, and aiding the theoretical understanding of deep networks. This article provides an in-depth and practical review of tensors and tensor methods in the context of representation learning and deep learning, with a particular focus on visual data analysis and computer vision applications. Concretely, besides fundamental work in tensor-based visual data analysis methods, we focus on recent developments that have brought on a gradual increase of tensor methods, especially in deep learning architectures, and their implications in computer vision applications. To further enable the newcomer to grasp such concepts quickly, we provide companion Python notebooks, covering key aspects of the paper and implementing them, step-by-step with TensorLy.

Key facts. Cited by 52 papers in the map (data through 14 Oct 2026). Builds on 76 of its 240 references that are in the map. First posted on arXiv 7 Jul 2021.

Builds on

76 of 240 references.

References · 76

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  3. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  4. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  5. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
  6. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,548
  7. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,845
  8. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,611
  9. 2017Non-local Neural Networkscited by 2,293
  10. 20194D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networkscited by 1,208
  11. 2016Stacked Hourglass Networks for Human Pose Estimationcited by 915
  12. 2017In-Datacenter Performance Analysis of a Tensor Processing Unitcited by 618
38 more
  1. 2019Implicit Regularization in Deep Matrix Factorizationcited by 405
  2. 2014Speeding up Convolutional Neural Networks with Low Rank Expansionscited by 284
  3. 2015Highway Networkscited by 260
  4. 2015Tensorizing Neural Networkscited by 250
  5. 2016Tensor Decomposition for Signal Processing and Machine Learningcited by 224
  6. 2014Speeding-up Convolutional Neural Networks Using Fine-tuned CP-Decompositioncited by 203
  7. 2015MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systemscited by 195
  8. 2015Compression of Deep Convolutional Neural Networks for Fast and Low Power Mobile Applicationscited by 162
  9. 2017Theoretical insights into the optimization landscape of over-parameterized shallow neural networkscited by 128
  10. 2016Tensor Ring Decompositioncited by 127
  11. 2016TensorLy: Tensor Learning in Pythoncited by 104
  12. 2018On the Power of Over-parametrization in Neural Networks with Quadratic Activationcited by 102
  13. 2018Shield: Fast, Practical Defense and Vaccination for Deep Learning using JPEG Compressioncited by 74
  14. 2020Stable Low-rank Tensor Decomposition for Compression of Convolutional Neural Networkcited by 74
  15. 2016Deep Multi-task Representation Learning: A Tensor Factorisation Approachcited by 69
  16. 2017More is Less: A More Complicated Network with Less Inference Complexitycited by 66
  17. 2014Bayesian CP Factorization of Incomplete Tensors with Automatic Rank Determinationcited by 65
  18. 2020Deep Polynomial Neural Networkscited by 63
  19. 2020Multi-Head Attention: Collaborate Instead of Concatenatecited by 60
  20. 2016A survey of sparse representation: algorithms and applicationscited by 58
  21. 2016Ultimate tensorization: compressing convolutional and FC layers alikecited by 56
  22. 2020Convolutional Tensor-Train LSTM for Spatio-temporal Learningcited by 55
  23. 2017Tensor Networks for Dimensionality Reduction and Large-Scale Optimizations. Part 2 Applications and Future Perspectivescited by 54
  24. 2016Higher-Order Factorization Machinescited by 45
  25. 2017Tensor Regression Networkscited by 44
  26. 2018TTHRESH: Tensor Compression for Multidimensional Visual Datacited by 42
  27. 2018Tensor Ring Decomposition with Rank Minimization on Latent Space: An Efficient Approach for Tensor Completioncited by 42
  28. 2019Factorized Higher-Order CNNs with an Application to Spatio-Temporal Emotion Estimationcited by 40
  29. 2020Pi-nets: Deep Polynomial Neural Networkscited by 39
  30. 2013Robust Low-rank Tensor Recovery: Models and Algorithmscited by 34
  31. 2016Convolutional Rectifier Networks as Generalized Tensor Decompositionscited by 32
  32. 2017CP-decomposition with Tensor Power Method for Convolutional Neural Networks Compressioncited by 30
  33. 2013Tucker Tensor Regression and Neuroimaging Analysiscited by 26
  34. 2019Einconv: Exploring Unexplored Tensor Network Decompositions for Convolutional Neural Networkscited by 25
  35. 2017Deep Learning and Quantum Entanglement: Fundamental Connections with Implications to Network Designcited by 24
  36. 2015Global Optimality in Tensor Factorization, Deep Learning, and Beyondcited by 22
  37. 2019T-Net: Parametrizing Fully Convolutional Nets with a Single High-Order Tensorcited by 22
  38. 2020Tensor Networks for Medical Image Classificationcited by 20

50 most cited of 76 All 76 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 2; 2022: 12; 2023: 9; 2024: 10; 2025: 13; 2026: 6.

52 papers.

Sort:

By area:

  1. 2022TensoRF: Tensorial Radiance Fieldscited by 1,280
  2. 2023Multi-Grid Tensorized Fourier Neural Operator for High-Resolution PDEscited by 68
  3. 2023Guaranteed Tensor Recovery Fused Low-rankness and Smoothnesscited by 41
  4. 2023Tensor Networks Meet Neural Networks: A Survey and Future Perspectivescited by 32
  5. 2024Bilinear MLPs enable weight-based mechanistic interpretabilitycited by 28
  6. 2024What is the Relationship between Tensor Factorizations and Circuits (and How Can We Exploit it)?cited by 22
  7. 2021Tensor Component Analysis for Interpreting the Latent Space of GANscited by 13
  8. 2022Rethinking Bayesian Learning for Data Analysis: The Art of Prior and Inference in Sparsity-Aware Modelingcited by 12
  9. 2022HEAT: Hardware-Efficient Automatic Tensor Decomposition for Transformer Compressioncited by 9
  10. 2023Guaranteed Approximation Bounds for Mixed-Precision Neural Operatorscited by 9
  11. 2023Generative Learning of Continuous Data by Tensor Networkscited by 9
  12. 2025Hadamard product in deep learning: Introduction, Advances and Challengescited by 9
38 more
  1. 2022tntorch: Tensor Network Learning with PyTorchcited by 7
  2. 2022TT-NF: Tensor Train Neural Fieldscited by 7
  3. 2021Reinforcement Learning in Factored Action Spaces using Tensor Decompositionscited by 6
  4. 2023Quantization Aware Factorization for Deep Neural Network Compressioncited by 6
  5. 2025TensorGRaD: Tensor Gradient Robust Decomposition for Memory-Efficient Neural Operator Trainingcited by 6
  6. 2022How to Train Unstable Looped Tensor Networkcited by 3
  7. 2022Position: Tensor Networks are a Valuable Asset for Green AIcited by 3
  8. 2022Many-body Approximation for Non-negative Tensorscited by 3
  9. 2022Positive unlabeled learning with tensor networkscited by 3
  10. 2024MIGS: Multi-Identity Gaussian Splatting via Tensor Decompositioncited by 3
  11. 2023Boosting Defect Detection in Manufacturing using Tensor Convolutional Neural Networkscited by 2
  12. 2024FedBRB: An Effective Solution to the Small-to-Large Scenario in Device-Heterogeneity Federated Learningcited by 2
  13. 2025Changing Base Without Losing Pace: A GPU-Efficient Alternative to MatMul in DNNscited by 2
  14. 2025Multiscale Tensor Summation Factorization as a New Neural Network Layer (MTS Layer) for Multidimensional Data Processingcited by 2
  15. 2025Low-Rank Tensor Decompositions for the Theory of Neural Networkscited by 2
  16. 2022Approximation of Images via Generalized Higher Order Singular Value Decomposition over Finite-dimensional Commutative Semisimple Algebracited by 1
  17. 2022A Comprehensive Review of Visual-Textual Sentiment Analysis from Social Media Networkscited by 1
  18. 2023Tensor Ring Optimized Quantum-Enhanced Tensor Neural Networkscited by 1
  19. 2024Generalized Least Squares Kernelized Tensor Factorizationcited by 1
  20. 2025tCURLoRA: Tensor CUR Decomposition Based Low-Rank Parameter Adaptation and Its Application in Medical Image Segmentationcited by 1
  21. 2025Multi-Scale Tensorial Summation and Dimensional Reduction Guided Neural Network for Edge Detectioncited by 1
  22. 2025Localized PCA-Net Neural Operators for Scalable Solution Reconstruction of Elliptic PDEscited by 1
  23. 2022Approximate Real Symmetric Tensor Rankcited by 0
  24. 2023Low-Multi-Rank High-Order Bayesian Robust Tensor Factorizationcited by 0
  25. 2024Exact Tensor Completion Powered by Slim Transformscited by 0
  26. 2024Improving LSH via Tensorized Random Projectioncited by 0
  27. 2024Compressing Structured Tensor Algebracited by 0
  28. 2024A Kernelizable Primal-Dual Formulation of the Multilinear Singular Value Decompositioncited by 0
  29. 2024Variational Bayesian Inference for Tensor Robust Principal Component Analysiscited by 0
  30. 2025Generative AI in Collaborative Academic Report Writing: Advantages, Disadvantages, and Ethical Considerationscited by 0
  31. 2025Accelerating Large-Scale Regularized High-Order Tensor Recoverycited by 0
  32. 2025ReLATE: Accelerating Tensor Decomposition via Safe and Efficient Learning of Sparse Encodingscited by 0
  33. 2025Learning Mixtures of Linear Dynamical Systems via Hybrid Tensor-EM Methodcited by 0
  34. 2025Data Understanding Survey: Pursuing Improved Dataset Characterization Via Tensor-based Methodscited by 0
  35. 2026Quaternion Nonlinear Transform-Induced Nuclear Norm for Low-Rank Tensor Completioncited by 0
  36. 2026fmxcoders: Factorized Masked Crosscoders for Cross-Layer Feature Discoverycited by 0
  37. 2026Transfer learning RGB models to hyperspectral images with trainable tensor decompositionscited by 0
  38. 2026Riemannian Gradient Descent for Low-Rank Architecturescited by 0

50 most cited of 52 All 52 (JSON)