# Matrix Completion via Nonsmooth Regularization of Fully Connected Neural Networks

arXiv: https://arxiv.org/abs/2403.10232 · HTML: https://mlsanity.com/paper/2403.10232/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 59 references that are in the map.
- First posted on arXiv 15 Mar 2024.

## Builds on (most cited first, 12 of 12)

- [Do Deep Nets Really Need to be Deep?](https://mlsanity.com/paper/1312.6184/) (2013)
- [Relatively-Smooth Convex Optimization by First-Order Methods, and Applications](https://mlsanity.com/paper/1610.05708/) (2016)
- [Pufferfish: Communication-efficient Models At No Extra Cost](https://mlsanity.com/paper/2103.03936/) (2021)
- [A globally convergent algorithm for nonconvex optimization based on block coordinate update](https://mlsanity.com/paper/1410.1386/) (2014)
- [Are wider nets better given the same number of parameters?](https://mlsanity.com/paper/2010.14495/) (2020)
- [Multi-block Bregman proximal alternating linearized minimization and its application to orthogonal nonnegative matrix factorization](https://mlsanity.com/paper/1908.01402/) (2019)
- [A block inertial Bregman proximal algorithm for nonsmooth nonconvex problems with application to symmetric nonnegative matrix tri-factorization](https://mlsanity.com/paper/2003.03963/) (2020)
- [Block Alternating Bregman Majorization Minimization with Extrapolation](https://mlsanity.com/paper/2107.04395/) (2021)
- [Matrix Completion via Non-Convex Relaxation and Adaptive Correlation Learning](https://mlsanity.com/paper/2203.02189/) (2022)
- [Regularizing Autoencoder-Based Matrix Completion Models via Manifold Learning](https://mlsanity.com/paper/1807.01798/) (2018)
- [Bregman Proximal Framework for Deep Linear Neural Networks](https://mlsanity.com/paper/1910.03638/) (2019)
- [Deep Learning Approach for Matrix Completion Using Manifold Learning](https://mlsanity.com/paper/2012.06063/) (2020)

## Abstract (arXiv)

Conventional matrix completion methods approximate the missing values by assuming the matrix to be low-rank, which leads to a linear approximation of missing values. It has been shown that enhanced performance could be attained by using nonlinear estimators such as deep neural networks. Deep fully connected neural networks (FCNNs), one of the most suitable architectures for matrix completion, suffer from over-fitting due to their high capacity, which leads to low generalizability. In this paper, we control over-fitting by regularizing the FCNN model in terms of the ℓ1 norm of intermediate representations and nuclear norm of weight matrices. As such, the resulting regularized objective function becomes nonsmooth and nonconvex, i.e., existing gradient-based methods cannot be applied to our model. We propose a variant of the proximal gradient method and investigate its convergence to a critical point. In the initial epochs of FCNN training, the regularization terms are ignored, and through epochs, the effect of that increases. The gradual addition of nonsmooth regularization terms is the main reason for the better performance of the deep neural network with nonsmooth regularization terms (DNN-NSR) algorithm. Our simulations indicate the superiority of the proposed algorithm in comparison with existing linear and nonlinear algorithms.