mlsanityData to 7 Oct 2026

Towards a General Purpose CNN for Long Range Dependencies in ND

arXiv 2206.03398 ↗First posted 7 Jun 2022cs.LG cs.CVcited by 16

Not on the map. Why?

Abstract (arXiv)

The use of Convolutional Neural Networks (CNNs) is widespread in Deep Learning due to a range of desirable model properties which result in an efficient and effective machine learning framework. However, performant CNN architectures must be tailored to specific tasks in order to incorporate considerations such as the input length, resolution, and dimentionality. In this work, we overcome the need for problem-specific CNN architectures with our Continuous Convolutional Neural Network (CCNN): a single CNN architecture equipped with continuous convolutional kernels that can be used for tasks on data of arbitrary resolution, dimensionality and length without structural changes. Continuous convolutional kernels model long range dependencies at every layer, and remove the need for downsampling layers and task-dependent depths needed in current CNN architectures. We show the generality of our approach by applying the same CCNN to a wide set of tasks on sequential (1D) and visual data (2D). Our CCNN performs competitively and often outperforms the current state-of-the-art across all tasks considered.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Builds on 27 of its 42 references that are in the map. First posted on arXiv 7 Jun 2022.

Builds on

27 of 42 references.

References · 27

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  4. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  5. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  6. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  7. 2014Going Deeper with Convolutionscited by 4,231
  8. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,590
  9. 2022A ConvNet for the 2020scited by 3,545
  10. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,424
  11. 2016Xception: Deep Learning with Depthwise Separable Convolutionscited by 2,320
  12. 2020Implicit Neural Representations with Periodic Activation Functionscited by 2,214
15 more
  1. 2016WaveNet: A Generative Model for Raw Audiocited by 1,710
  2. 2018An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modelingcited by 1,447
  3. 2020On Layer Normalization in the Transformer Architecturecited by 906
  4. 2018Speech Commands: A Dataset for Limited-Vocabulary Speech Recognitioncited by 741
  5. 2020Long Range Arena: A Benchmark for Efficient Transformerscited by 696
  6. 2018PointConv: Deep Convolutional Networks on 3D Point Cloudscited by 656
  7. 2020HiPPO: Recurrent Memory with Optimal Polynomial Projectionscited by 600
  8. 2017SchNet: A continuous-filter convolutional neural network for modeling quantum interactionscited by 569
  9. 2020Neural Controlled Differential Equations for Irregular Time Seriescited by 501
  10. 2015A Simple Way to Initialize Recurrent Networks of Rectified Linear Unitscited by 130
  11. 2021CKConv: Continuous Kernel Convolution For Sequential Datacited by 117
  12. 2017Dilated Recurrent Neural Networkscited by 83
  13. 2021FlexConv: Continuous Kernel Convolutions with Differentiable Kernel Sizescited by 66
  14. 2014Rigid-Motion Scattering for Texture Classificationcited by 52
  15. 2021Exploiting Redundancy: Separable Group Convolutional Networks on Lie Groupscited by 22

Built on by

Cited by year

Citing papers per year: 2022: 5; 2023: 3; 2024: 6; 2025: 2.

16 papers.

Sort:

By area:

  1. 2022Simplified State Space Layers for Sequence Modelingcited by 776
  2. 2022Towards Multi-spatiotemporal-scale Generalized PDE Modelingcited by 190
  3. 2022Mega: Moving Average Equipped Gated Attentioncited by 178
  4. 2022More ConvNets in the 2020s: Scaling up Kernels Beyond 51x51 using Sparsitycited by 142
  5. 2024Mamba-360: Survey of State Space Models as Transformer Alternative for Long Sequence Modelling: Methods, Applications, and Challengescited by 63
  6. 2024ELA: Efficient Local Attention for Deep Convolutional Neural Networkscited by 8
  7. 2024A Diagonal Structured State Space Model on Loihi 2 for Efficient Streaming Sequence Processingcited by 6
  8. 2022A Continuous Convolutional Trainable Filter for Modelling Unstructured Datacited by 5
  9. 2023DNArch: Learning Convolutional Neural Architectures by Backpropagationcited by 3
  10. 2023Convolutional Networks with Oriented 1D Kernelscited by 3
  11. 2024Towards Better Multi-head Attention via Channel-wise Sample Permutationcited by 2
  12. 2025Myosotis: structured computation for attention like layercited by 2
4 more
  1. 2023A Quantitative Review on Language Model Efficiency Researchcited by 0
  2. 2024HyperZ·Z·W Operator Connects Slow-Fast Networks for Full Context Interactioncited by 0
  3. 2024Efficient State Space Model via Fast Tensor Convolution and Block Diagonalizationcited by 0
  4. 2025CNN-BiLSTM for sustainable and non-invasive COVID-19 detection via salivary ATR-FTIR spectroscopycited by 0