# Beyond ℓ1 sparse coding in V1

arXiv: https://arxiv.org/abs/2301.10002 · HTML: https://mlsanity.com/paper/2301.10002/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Jan 2023.

## Built on by (most cited first, 4 of 4)

- [Archetypal SAE: Adaptive and Stable Dictionary Learning for Concept Extraction in Large Vision Models](https://mlsanity.com/paper/2502.12892/) (2025)
- [Cross-Modal Redundancy and the Geometry of Vision-Language Embeddings](https://mlsanity.com/paper/2602.06218/) (2026)
- [WARP-LCA: Efficient Convolutional Sparse Coding with Locally Competitive Algorithm](https://mlsanity.com/paper/2410.18794/) (2024)
- [A Unifying Framework for Concept-Based Representational Similarity](https://mlsanity.com/paper/2606.09653/) (2026)

## Abstract (arXiv)

Growing evidence indicates that only a sparse subset from a pool of sensory neurons is active for the encoding of visual stimuli at any instant in time. Traditionally, to replicate such biological sparsity, generative models have been using the ℓ1 norm as a penalty due to its convexity, which makes it amenable to fast and simple algorithmic solvers. In this work, we use biological vision as a test-bed and show that the soft thresholding operation associated to the use of the ℓ1 norm is highly suboptimal compared to other functions suited to approximating ℓq with 0 ≤ q < 1  (including recently proposed Continuous Exact relaxations), both in terms of performance and in the production of features that are akin to signatures of the primary visual cortex. We show that ℓ1 sparsity produces a denser code or employs a pool with more neurons, i.e. has a higher degree of overcompleteness, in order to maintain the same reconstruction error as the other methods considered. For all the penalty functions tested, a subset of the neurons develop orientation selectivity similarly to V1 neurons. When their code is sparse enough, the methods also develop receptive fields with varying functionalities, another signature of V1. Compared to other methods, soft thresholding achieves this level of sparsity at the expense of much degraded reconstruction performance, that more likely than not is not acceptable in biological vision. Our results indicate that V1 uses a sparsity inducing regularization that is closer to the ℓ0 pseudo-norm rather than to the ℓ1 norm.