# Frank-Wolfe Network: An Interpretable Deep Structure for Non-Sparse Coding

arXiv: https://arxiv.org/abs/1802.10252 · HTML: https://mlsanity.com/paper/1802.10252/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Feb 2018.

## Built on by (most cited first, 3 of 3)

- [Learning to Optimize: A Primer and A Benchmark](https://mlsanity.com/paper/2103.12828/) (2021)
- [Stochastic Unrolled Federated Learning](https://mlsanity.com/paper/2305.15371/) (2023)
- [Hybrid ISTA: Unfolding ISTA With Convergence Guarantees Using Free-Form Deep Neural Networks](https://mlsanity.com/paper/2204.11640/) (2022)

## Abstract (arXiv)

The problem of Lp-norm constrained coding is to convert signal into code that lies inside an Lp-ball and most faithfully reconstructs the signal. Previous works under the name of sparse coding considered the cases of L0 and L1 norms. The cases with p>1 values, i.e. non-sparse coding studied in this paper, remain a difficulty. We propose an interpretable deep structure namely Frank-Wolfe Network (F-W Net), whose architecture is inspired by unrolling and truncating the Frank-Wolfe algorithm for solving an Lp-norm constrained problem with p≥ 1. We show that the Frank-Wolfe solver for the Lp-norm constraint leads to a novel closed-form nonlinear unit, which is parameterized by p and termed poolp. The poolp unit links the conventional pooling, activation, and normalization operations, making F-W Net distinct from existing deep networks either heuristically designed or converted from projected gradient descent algorithms. We further show that the hyper-parameter p can be made learnable instead of pre-chosen in F-W Net, which gracefully solves the non-sparse coding problem even with unknown p. We evaluate the performance of F-W Net on an extensive range of simulations as well as the task of handwritten digit recognition, where F-W Net exhibits strong learning capability. We then propose a convolutional version of F-W Net, and apply the convolutional F-W Net into image denoising and super-resolution tasks, where F-W Net all demonstrates impressive effectiveness, flexibility, and robustness.