# Scaling the Scattering Transform: Deep Hybrid Networks

arXiv: https://arxiv.org/abs/1703.08961 · HTML: https://mlsanity.com/paper/1703.08961/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Mar 2017.

## Built on by (most cited first, 30 of 35)

- [Learning Operators with Coupled Attention](https://mlsanity.com/paper/2201.01032/) (2022)
- [Wave-ViT: Unifying Wavelet and Transformers for Visual Representation Learning](https://mlsanity.com/paper/2207.04978/) (2022)
- [Resolution learning in deep convolutional networks using scale-space theory](https://mlsanity.com/paper/2106.03412/) (2021)
- [Parametric Scattering Networks](https://mlsanity.com/paper/2107.09539/) (2021)
- [Tune It or Don't Use It: Benchmarking Data-Efficient Image Classification](https://mlsanity.com/paper/2108.13122/) (2021)
- [The Unreasonable Effectiveness of Patches in Deep Convolutional Kernels Methods](https://mlsanity.com/paper/2101.07528/) (2021)
- [Frequency learning for structured CNN filters with Gaussian fractional derivatives](https://mlsanity.com/paper/2111.06660/) (2021)
- [Image Classification with Small Datasets: Overview and Benchmark](https://mlsanity.com/paper/2212.12478/) (2022)
- [Riesz feature representation: scale equivariant scattering network for classification tasks](https://mlsanity.com/paper/2307.08467/) (2023)
- [Learnable Discrete Wavelet Pooling (LDW-Pooling) For Convolutional Networks](https://mlsanity.com/paper/2109.06638/) (2021)
- [Wavelet Regularization Benefits Adversarial Training](https://mlsanity.com/paper/2206.03727/) (2022)
- [Faster Convergence in Deep-Predictive-Coding Networks to Learn Deeper Representations](https://mlsanity.com/paper/2101.06848/) (2021)
- [Hybrid BYOL-ViT: Efficient approach to deal with small datasets](https://mlsanity.com/paper/2111.04845/) (2021)
- [Resource-Efficient Invariant Networks: Exponential Gains by Unrolled Optimization](https://mlsanity.com/paper/2203.05006/) (2022)
- [On the Shift Invariance of Max Pooling Feature Maps in Convolutional Neural Networks](https://mlsanity.com/paper/2209.11740/) (2022)
- [Infinite Class Mixup](https://mlsanity.com/paper/2305.10293/) (2023)
- [Manifold Filter-Combine Networks](https://mlsanity.com/paper/2307.04056/) (2023)
- [Representation Learning in a Decomposed Encoder Design for Bio-inspired Hebbian Learning](https://mlsanity.com/paper/2401.08603/) (2023)
- [Signature-Graph Networks](https://mlsanity.com/paper/2110.11551/) (2021)
- [GCCN: Global Context Convolutional Network](https://mlsanity.com/paper/2110.11664/) (2021)
- [Efficient Hybrid Network: Inducting Scattering Features](https://mlsanity.com/paper/2203.15392/) (2022)
- [DePF: A Novel Fusion Approach based on Decomposition Pooling for Infrared and Visible Images](https://mlsanity.com/paper/2305.17376/) (2023)
- [No Data Augmentation? Alternative Regularizations for Effective Training on Small Datasets](https://mlsanity.com/paper/2309.01694/) (2023)
- [Local convergence of simultaneous min-max algorithms to differential equilibrium on Riemannian manifold](https://mlsanity.com/paper/2405.13392/) (2024)
- [Uncertainty-Aware Decomposed Hybrid Networks](https://mlsanity.com/paper/2503.19096/) (2025)
- [The Maximal Overlap Discrete Wavelet Scattering Transform and Its Application in Classification Tasks](https://mlsanity.com/paper/2506.12039/) (2025)
- [VDW-GNNs: Vector diffusion wavelets for geometric graph neural networks](https://mlsanity.com/paper/2510.01022/) (2025)
- [Scale-Equivariant Generative Forecasting: Weight-Tied Dilated Convolutions, Wavelet Scattering Inputs, and Spectral-Consistency Training for Self-Similar Time Series](https://mlsanity.com/paper/2605.17582/) (2026)
- [CXR-Net: An Artificial Intelligence Pipeline for Quick Covid-19 Screening of Chest X-Rays](https://mlsanity.com/paper/2103.00087/) (2021)
- [A Structurally Regularized Convolutional Neural Network for Image Classification using Wavelet-based SubBand Decomposition](https://mlsanity.com/paper/2103.01823/) (2021)

## Abstract (arXiv)

We use the scattering network as a generic and fixed ini-tialization of the first layers of a supervised hybrid deep network. We show that early layers do not necessarily need to be learned, providing the best results to-date with pre-defined representations while being competitive with Deep CNNs. Using a shallow cascade of 1 x 1 convolutions, which encodes scattering coefficients that correspond to spatial windows of very small sizes, permits to obtain AlexNet accuracy on the imagenet ILSVRC2012. We demonstrate that this local encoding explicitly learns invariance w.r.t. rotations. Combining scattering networks with a modern ResNet, we achieve a single-crop top 5 error of 11.4% on imagenet ILSVRC2012, comparable to the Resnet-18 architecture, while utilizing only 10 layers. We also find that hybrid architectures can yield excellent performance in the small sample regime, exceeding their end-to-end counterparts, through their ability to incorporate geometrical priors. We demonstrate this on subsets of the CIFAR-10 dataset and on the STL-10 dataset.