mlsanityData to 14 Oct 2026

Scaling the Scattering Transform: Deep Hybrid Networks

arXiv 1703.08961 ↗First posted 27 Mar 2017cs.CV cs.LGcited by 35

Not on the map. Why?

Abstract (arXiv)

We use the scattering network as a generic and fixed ini-tialization of the first layers of a supervised hybrid deep network. We show that early layers do not necessarily need to be learned, providing the best results to-date with pre-defined representations while being competitive with Deep CNNs. Using a shallow cascade of 1 x 1 convolutions, which encodes scattering coefficients that correspond to spatial windows of very small sizes, permits to obtain AlexNet accuracy on the imagenet ILSVRC2012. We demonstrate that this local encoding explicitly learns invariance w.r.t. rotations. Combining scattering networks with a modern ResNet, we achieve a single-crop top 5 error of 11.4% on imagenet ILSVRC2012, comparable to the Resnet-18 architecture, while utilizing only 10 layers. We also find that hybrid architectures can yield excellent performance in the small sample regime, exceeding their end-to-end counterparts, through their ability to incorporate geometrical priors. We demonstrate this on subsets of the CIFAR-10 dataset and on the STL-10 dataset.

Key facts. Cited by 35 papers in the map (data through 14 Oct 2026). First posted on arXiv 27 Mar 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 13; 2022: 9; 2023: 6; 2024: 1; 2025: 4; 2026: 2.

35 papers.

Sort:

  1. 2022Learning Operators with Coupled Attentioncited by 90
  2. 2022Wave-ViT: Unifying Wavelet and Transformers for Visual Representation Learningcited by 78
  3. 2021Resolution learning in deep convolutional networks using scale-space theorycited by 26
  4. 2021Parametric Scattering Networkscited by 16
  5. 2021Tune It or Don't Use It: Benchmarking Data-Efficient Image Classificationcited by 12
  6. 2021The Unreasonable Effectiveness of Patches in Deep Convolutional Kernels Methodscited by 8
  7. 2021Frequency learning for structured CNN filters with Gaussian fractional derivativescited by 6
  8. 2022Image Classification with Small Datasets: Overview and Benchmarkcited by 4
  9. 2023Riesz feature representation: scale equivariant scattering network for classification taskscited by 4
  10. 2021Learnable Discrete Wavelet Pooling (LDW-Pooling) For Convolutional Networkscited by 3
  11. 2022Wavelet Regularization Benefits Adversarial Trainingcited by 3
  12. 2021Faster Convergence in Deep-Predictive-Coding Networks to Learn Deeper Representationscited by 2
23 more
  1. 2021Hybrid BYOL-ViT: Efficient approach to deal with small datasetscited by 2
  2. 2022Resource-Efficient Invariant Networks: Exponential Gains by Unrolled Optimizationcited by 2
  3. 2022On the Shift Invariance of Max Pooling Feature Maps in Convolutional Neural Networkscited by 2
  4. 2023Infinite Class Mixupcited by 2
  5. 2023Manifold Filter-Combine Networkscited by 2
  6. 2023Representation Learning in a Decomposed Encoder Design for Bio-inspired Hebbian Learningcited by 2
  7. 2021Signature-Graph Networkscited by 1
  8. 2021GCCN: Global Context Convolutional Networkcited by 1
  9. 2022Efficient Hybrid Network: Inducting Scattering Featurescited by 1
  10. 2023DePF: A Novel Fusion Approach based on Decomposition Pooling for Infrared and Visible Imagescited by 1
  11. 2023No Data Augmentation? Alternative Regularizations for Effective Training on Small Datasetscited by 1
  12. 2024Local convergence of simultaneous min-max algorithms to differential equilibrium on Riemannian manifoldcited by 1
  13. 2025Uncertainty-Aware Decomposed Hybrid Networkscited by 1
  14. 2025The Maximal Overlap Discrete Wavelet Scattering Transform and Its Application in Classification Taskscited by 1
  15. 2025VDW-GNNs: Vector diffusion wavelets for geometric graph neural networkscited by 1
  16. 2026Scale-Equivariant Generative Forecasting: Weight-Tied Dilated Convolutions, Wavelet Scattering Inputs, and Spectral-Consistency Training for Self-Similar Time Seriescited by 1
  17. 2021CXR-Net: An Artificial Intelligence Pipeline for Quick Covid-19 Screening of Chest X-Rayscited by 0
  18. 2021A Structurally Regularized Convolutional Neural Network for Image Classification using Wavelet-based SubBand Decompositioncited by 0
  19. 2021Concurrent Discrimination and Alignment for Self-Supervised Feature Learningcited by 0
  20. 2022Survey on Self-supervised Representation Learning Using Image Transformationscited by 0
  21. 2022From CNNs to Shift-Invariant Twin Models Based on Complex Waveletscited by 0
  22. 2025Freqformer: Image-Demoir'eing Transformer via Effective Frequency Decompositioncited by 0
  23. 2026Transversal Pooling Neural Networkscited by 0