# Cascaded Compressed Sensing Networks: A Reversible Architecture for Layerwise Learning

arXiv: https://arxiv.org/abs/2110.10379 · HTML: https://mlsanity.com/paper/2110.10379/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 34 references that are in the map.
- First posted on arXiv 20 Oct 2021.

## Builds on (most cited first, 12 of 12)

- [Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Data](https://mlsanity.com/paper/1703.11008/) (2017)
- [Breaking the Curse of Dimensionality with Convex Neural Networks](https://mlsanity.com/paper/1412.8690/) (2014)
- [Difference Target Propagation](https://mlsanity.com/paper/1412.7525/) (2014)
- [Training Neural Networks with Local Error Signals](https://mlsanity.com/paper/1901.06656/) (2019)
- [Greedy Layerwise Learning Can Scale to ImageNet](https://mlsanity.com/paper/1812.11446/) (2018)
- [Assessing the Scalability of Biologically-Motivated Deep Learning Algorithms and Architectures](https://mlsanity.com/paper/1807.04587/) (2018)
- [How Auto-Encoders Could Provide Credit Assignment in Deep Networks via Target Propagation](https://mlsanity.com/paper/1407.7906/) (2014)
- [A survey of sparse representation: algorithms and applications](https://mlsanity.com/paper/1602.07017/) (2016)
- [Generalization in Deep Networks: The Role of Distance from Initialization](https://mlsanity.com/paper/1901.01672/) (2019)
- [Sparse Modeling for Image and Vision Processing](https://mlsanity.com/paper/1411.3230/) (2014)
- [Deep Network Classification by Scattering and Homotopy Dictionary Learning](https://mlsanity.com/paper/1910.03561/) (2019)
- [Why Layer-Wise Learning is Hard to Scale-up and a Possible Solution via Accelerated Downsampling](https://mlsanity.com/paper/2010.08038/) (2020)

## Abstract (arXiv)

Recently, the method that learns networks layer by layer has attracted increasing interest for its ease of analysis. For the method, the main challenge lies in deriving an optimization target for each layer by inversely propagating the global target of the network. The propagation problem is ill posed, due to involving the inversion of nonlinear activations from lowdimensional to high-dimensional spaces. To address the problem, the existing solution is to learn an auxiliary network to specially propagate the target. However, the network lacks stability, and moreover, it results in higher complexity for network learning. In the letter, we show that target propagation could be achieved by modeling the network s each layer with compressed sensing, without the need of auxiliary networks. Experiments show that the proposed method could achieve better performance than the auxiliary network-based method.