mlsanityData to 7 Oct 2026

Modelling Long Range Dependencies in ND: From Task-Specific to a General Purpose CNN

arXiv 2301.10540 ↗First posted 25 Jan 2023cs.CVcited by 18

Not on the map. Why?

Abstract (arXiv)

Performant Convolutional Neural Network (CNN) architectures must be tailored to specific tasks in order to consider the length, resolution, and dimensionality of the input data. In this work, we tackle the need for problem-specific CNN architectures. We present the Continuous Convolutional Neural Network (CCNN): a single CNN able to process data of arbitrary resolution, dimensionality and length without any structural changes. Its key component are its continuous convolutional kernels which model long-range dependencies at every layer, and thus remove the need of current CNN architectures for task-dependent downsampling and depths. We showcase the generality of our method by using the same architecture for tasks on sequential (1rm D), visual (2rm D) and point-cloud (3rm D) data. Our CCNN matches and often outperforms the current state-of-the-art across all tasks considered.

Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). Builds on 48 of its 65 references that are in the map. First posted on arXiv 25 Jan 2023.

Builds on

48 of 65 references.

References · 48

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  4. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  5. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  6. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  7. 2014Going Deeper with Convolutionscited by 4,231
  8. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,590
  9. 2022A ConvNet for the 2020scited by 3,545
  10. 2017Neural Message Passing for Quantum Chemistrycited by 3,460
  11. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,424
  12. 2020Implicit Neural Representations with Periodic Activation Functionscited by 2,214
36 more
  1. 2020Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domainscited by 2,170
  2. 2016WaveNet: A Generative Model for Raw Audiocited by 1,710
  3. 2017Deformable Convolutional Networkscited by 1,481
  4. 2018An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modelingcited by 1,447
  5. 2019KPConv: Flexible and Deformable Convolution for Point Cloudscited by 1,202
  6. 2021Perceiver: General Perception with Iterative Attentioncited by 1,089
  7. 2020On Layer Normalization in the Transformer Architecturecited by 906
  8. 2018Speech Commands: A Dataset for Limited-Vocabulary Speech Recognitioncited by 741
  9. 2020Long Range Arena: A Benchmark for Efficient Transformerscited by 696
  10. 2018PointConv: Deep Convolutional Networks on 3D Point Cloudscited by 656
  11. 2021Combining Recurrent, Convolutional, and Continuous-time Models with Linear State-Space Layerscited by 607
  12. 2017SchNet: A continuous-filter convolutional neural network for modeling quantum interactionscited by 569
  13. 2020Neural Controlled Differential Equations for Irregular Time Seriescited by 501
  14. 2019General E(2)-Equivariant Steerable CNNscited by 441
  15. 20183D Steerable CNNs: Learning Rotationally Equivariant Features in Volumetric Datacited by 348
  16. 2016Dynamic Filter Networkscited by 318
  17. 2016Harmonic Networks: Deep Translation and Rotation Equivariancecited by 293
  18. 2018SpiderCNN: Deep Learning on Point Sets with Parameterized Convolutional Filterscited by 261
  19. 2020Generalizing Convolutional Neural Networks for Equivariance to Lie Groups on Arbitrary Continuous Datacited by 248
  20. 2018Deep Neural Networks Motivated by Partial Differential Equationscited by 187
  21. 2021Deep Parametric Continuous Convolutional Neural Networkscited by 161
  22. 2017Pointwise Convolutional Neural Networkscited by 126
  23. 2021CKConv: Continuous Kernel Convolution For Sequential Datacited by 117
  24. 2019Deep Scale-spaces: Equivariance Over Scalecited by 113
  25. 2019Scale-Equivariant Steerable Networkscited by 110
  26. 2019B-Spline CNNs on Lie Groupscited by 97
  27. 2021FlexConv: Continuous Kernel Convolutions with Differentiable Kernel Sizescited by 66
  28. 2020Group Equivariant Stand-Alone Self-Attention For Visioncited by 63
  29. 2014Rigid-Motion Scattering for Texture Classificationcited by 52
  30. 2022S4ND: Modeling Images and Videos as Multidimensional Signals Using State Spacescited by 41
  31. 2021DISCO: accurate Discrete Scale Convolutionscited by 35
  32. 2022Learning strides in convolutional neural networkscited by 25
  33. 2017Active Convolution: Learning the Shape of Convolution for Image Classificationcited by 24
  34. 2021Exploiting Redundancy: Separable Group Convolutional Networks on Lie Groupscited by 22
  35. 2017Raw Waveform-based Audio Classification Using Sample-level CNN Architecturescited by 12
  36. 2021Log-Polar Space Convolution for Convolutional Neural Networkscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 6; 2024: 9; 2025: 2; 2026: 1.

18 papers.

Sort:

By area:

  1. 2023Facing Off World Model Backbones: RNNs, Transformers, and S4cited by 31
  2. 20232-D SSM: A General Spatial Layer for Visual Transformerscited by 18
  3. 2024Slot State Space Modelscited by 17
  4. 2024LoG-VMamba: Local-Global Vision Mamba for Medical Image Segmentationcited by 8
  5. 2024Orchid: Flexible and Data-Dependent Convolution for Sequence Modelingcited by 6
  6. 2023Multi-Dimensional Hyena for Spatial Inductive Biascited by 5
  7. 2024Spectral Convolutional Conditional Neural Processescited by 4
  8. 2024CKGConv: General Graph Convolution with Continuous Kernelscited by 4
  9. 2023DNArch: Learning Convolutional Neural Architectures by Backpropagationcited by 3
  10. 2025From Embeddings to Accuracy: Comparing Foundation Models for Radiographic Classificationcited by 3
  11. 2023Continuous Convolutional Neural Networks for Disruption Prediction in Nuclear Fusion Plasmascited by 2
  12. 2024FISBe: A real-world benchmark dataset for instance segmentation of long-range thin filamentous structurescited by 2
6 more
  1. 2024Reparameterized Multi-Resolution Convolutions for Long Sequence Modellingcited by 2
  2. 2023Learned Gridification for Efficient Point Cloud Processingcited by 1
  3. 2024Explainable Global Wildfire Prediction Models using Graph Neural Networkscited by 1
  4. 2024Scaling Continuous Kernels with Sparse Fourier Domain Learningcited by 0
  5. 2025Conditional Clifford-Steerable CNNs for PDE Modelingcited by 0
  6. 2026Revisiting Neural Processes via Fourier Transform and Volterra Seriescited by 0