# Effect of top-down connections in Hierarchical Sparse Coding

arXiv: https://arxiv.org/abs/2002.00892 · HTML: https://mlsanity.com/paper/2002.00892/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Feb 2020.

## Built on by (most cited first, 6 of 6)

- [Diversity vs. Recognizability: Human-like generalization in one-shot generative models](https://mlsanity.com/paper/2205.10370/) (2022)
- [Faster Convergence in Deep-Predictive-Coding Networks to Learn Deeper Representations](https://mlsanity.com/paper/2101.06848/) (2021)
- [Learning and Inference in Sparse Coding Models with Langevin Dynamics](https://mlsanity.com/paper/2204.11150/) (2022)
- [OscNet: Machine Learning on CMOS Oscillator Networks](https://mlsanity.com/paper/2502.07192/) (2025)
- [How Robust Are Energy-Based Models Trained With Equilibrium Propagation?](https://mlsanity.com/paper/2401.11543/) (2024)
- [Toward a mechanistic understanding of inference in visual cortex and diffusion models](https://mlsanity.com/paper/2607.15693/) (2026)

## Abstract (arXiv)

Hierarchical Sparse Coding (HSC) is a powerful model to efficiently represent multi-dimensional, structured data such as images. The simplest solution to solve this computationally hard problem is to decompose it into independent layer-wise subproblems. However, neuroscientific evidence would suggest inter-connecting these subproblems as in the Predictive Coding (PC) theory, which adds top-down connections between consecutive layers. In this study, a new model called 2-Layers Sparse Predictive Coding (2L-SPC) is introduced to assess the impact of this inter-layer feedback connection. In particular, the 2L-SPC is compared with a Hierarchical Lasso (Hi-La) network made out of a sequence of independent Lasso layers. The 2L-SPC and the 2-layers Hi-La networks are trained on 4 different databases and with different sparsity parameters on each layer. First, we show that the overall prediction error generated by 2L-SPC is lower thanks to the feedback mechanism as it transfers prediction error between layers. Second, we demonstrate that the inference stage of the 2L-SPC is faster to converge than for the Hi-La model. Third, we show that the 2L-SPC also accelerates the learning process. Finally, the qualitative analysis of both models dictionaries, supported by their activation probability, show that the 2L-SPC features are more generic and informative.