mlsanityData to 7 Oct 2026

Multilinear Operator Networks

arXiv 2401.17992 ↗First posted 31 Jan 2024cs.CV cs.LGcited by 12

Not on the map. Why?

Abstract (arXiv)

Despite the remarkable capabilities of deep neural networks in image recognition, the dependence on activation functions remains a largely unexplored area and has yet to be eliminated. On the other hand, Polynomial Networks is a class of models that does not require activation functions, but have yet to perform on par with modern architectures. In this work, we aim close this gap and propose MONet, which relies solely on multilinear operators. The core layer of MONet, called Mu-Layer, captures multiplicative interactions of the elements of the input token. MONet captures high-degree interactions of the input elements and we demonstrate the efficacy of our approach on a series of image recognition and scientific computing benchmarks. The proposed model outperforms prior polynomial networks and performs on par with modern architectures. We believe that MONet can inspire further research on models that use entirely multilinear operations.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 47 of its 65 references that are in the map. First posted on arXiv 31 Jan 2024.

Builds on

47 of 65 references.

References · 47

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  5. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  6. 2016Layer Normalizationcited by 5,579
  7. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  8. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  9. 2016Feature Pyramid Networks for Object Detectioncited by 5,021
  10. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,845
  11. 2017Squeeze-and-Excitation Networkscited by 4,763
  12. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
35 more
  1. 2018Neural Ordinary Differential Equationscited by 3,733
  2. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,590
  3. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,923
  4. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,875
  5. 2017Non-local Neural Networkscited by 2,290
  6. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,960
  7. 2021MLP-Mixer: An all-MLP Architecture for Visioncited by 1,441
  8. 2018AutoAugment: Learning Augmentation Policies from Datacited by 728
  9. 2019Selective Kernel Networkscited by 401
  10. 2019Augmented Neural ODEscited by 396
  11. 2021Pay Attention to MLPscited by 367
  12. 2019Attention Augmented Convolutional Networkscited by 363
  13. 2021ResMLP: Feedforward networks for image classification with data-efficient trainingcited by 338
  14. 2021DyTox: Transformers for Continual Learning with DYnamic TOken eXpansioncited by 291
  15. 2020MedMNIST Classification Decathlon: A Lightweight AutoML Benchmark for Medical Image Analysiscited by 206
  16. 2022Patches Are All You Need?cited by 201
  17. 2018A2-Nets: Double Attention Networkscited by 136
  18. 2020Disentangled Non-Local Neural Networkscited by 132
  19. 2021CycleMLP: A MLP-like Architecture for Dense Predictioncited by 126
  20. 2021Vision Permutator: A Permutable MLP-Like Architecture for Visual Recognitioncited by 103
  21. 2018Benchmarking Neural Network Robustness to Common Corruptions and Surface Variationscited by 99
  22. 2021S2-MLP: Spatial-Shift MLP Architecture for Visioncited by 83
  23. 2021Hire-MLP: Vision MLP via Hierarchical Rearrangementcited by 66
  24. 2021Do You Even Need Attention? A Stack of Feed-Forward Layers Does Surprisingly Well on ImageNetcited by 61
  25. 2020Pi-nets: Deep Polynomial Neural Networkscited by 39
  26. 2022Scalable Interpretability via Polynomialscited by 25
  27. 2021S2-MLPv2: Improved Spatial-Shift MLP Architecture for Visioncited by 24
  28. 2022Interpretable Polynomial Neural Ordinary Differential Equationscited by 24
  29. 2023Polynomial Neural Fields for Subband Decomposition and Manipulationcited by 22
  30. 2021Augmenting Deep Classifiers with Polynomial Neural Networkscited by 21
  31. 2020Multilinear Latent Conditioning for Generating Unseen Attribute Combinationscited by 14
  32. 2023Regularization of polynomial networks for image recognitioncited by 13
  33. 2021CoPE: Conditional image generation using Polynomial Expansionscited by 12
  34. 2021Poly-NL: Linear Complexity Non-local Layers with Polynomialscited by 5
  35. 2022BiMLP: Compact Binary Architectures for Vision Multi-Layer Perceptronscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 6; 2025: 4; 2026: 2.

12 papers.

  1. 2024Multilinear Mixture of Experts: Scalable Expert Specialization through Factorizationcited by 28
  2. 2025A Probabilistic Neuro-symbolic Layer for Algebraic Constraint Satisfactioncited by 11
  3. 2024AdaNCA: Neural Cellular Automata As Adaptors For More Robust Vision Transformercited by 4
  4. 2025Deep Tree Tensor Networkscited by 4
  5. 2024A Tensor Decomposition Perspective on Second-order RNNscited by 3
  6. 2024PNeRV: A Polynomial Neural Representation for Videoscited by 2
  7. 2024SpiralMLP: A Lightweight Vision MLP Architecturecited by 1
  8. 2024Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximationcited by 1
  9. 2025Unified Spatiotemporal Physics-Informed Learning (USPIL): A Framework for Modeling Complex Predator-Prey Dynamicscited by 1
  10. 2025Canonical Latent Representations in Conditional Diffusion Modelscited by 0
  11. 2026On the Role of Depth in the Expressivity of RNNscited by 0
  12. 2026Activation-Free Backbones for Image Recognition: Polynomial Alternatives within MetaFormer-Style Vision Modelscited by 0