# An Analytic End-to-End Deep Learning Algorithm based on Collaborative Learning

arXiv: https://arxiv.org/abs/2305.18594 · HTML: https://mlsanity.com/paper/2305.18594/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 7 of its 13 references that are in the map.
- First posted on arXiv 26 May 2023.

## Builds on (most cited first, 7 of 7)

- [A Convergence Theory for Deep Learning via Over-Parameterization](https://mlsanity.com/paper/1811.03962/) (2018)
- [Gradient Descent Finds Global Minima of Deep Neural Networks](https://mlsanity.com/paper/1811.03804/) (2018)
- [Stochastic Gradient Descent Optimizes Over-parameterized Deep ReLU Networks](https://mlsanity.com/paper/1811.08888/) (2018)
- [Collaborative Deep Learning for Recommender Systems](https://mlsanity.com/paper/1409.2944/) (2014)
- [Collaborative Learning for Deep Neural Networks](https://mlsanity.com/paper/1805.11761/) (2018)
- [Effects of Depth, Width, and Initialization: A Convergence Analysis of Layer-wise Training for Deep Linear Neural Networks](https://mlsanity.com/paper/1910.05874/) (2019)
- [An Analytic Layer-wise Deep Learning Framework with Applications to Robotics](https://mlsanity.com/paper/2102.03705/) (2021)

## Abstract (arXiv)

In most control applications, theoretical analysis of the systems is crucial in ensuring stability or convergence, so as to ensure safe and reliable operations and also to gain a better understanding of the systems for further developments. However, most current deep learning methods are black-box approaches that are more focused on empirical studies. Recently, some results have been obtained for convergence analysis of end-to end deep learning based on non-smooth ReLU activation functions, which may result in chattering for control tasks. This paper presents a convergence analysis for end-to-end deep learning of fully connected neural networks (FNN) with smooth activation functions. The proposed method therefore avoids any potential chattering problem, and it also does not easily lead to gradient vanishing problems. The proposed End-to-End algorithm trains multiple two-layer fully connected networks concurrently and collaborative learning can be used to further combine their strengths to improve accuracy. A classification case study based on fully connected networks and MNIST dataset was done to demonstrate the performance of the proposed approach. Then an online kinematics control task of a UR5e robot arm was performed to illustrate the regression approximation and online updating ability of our algorithm.