# Towards Deep Robot Learning with Optimizer applicable to Non-stationary Problems

arXiv: https://arxiv.org/abs/2007.15890 · HTML: https://mlsanity.com/paper/2007.15890/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Jul 2020.

## Built on by (most cited first, 3 of 3)

- [Need is All You Need: Homeostatic Neural Networks Adapt to Concept Shift](https://mlsanity.com/paper/2205.08645/) (2022)
- [Proximal Policy Optimization with Adaptive Threshold for Symmetric Relative Density Ratio](https://mlsanity.com/paper/2203.09809/) (2022)
- [Optimization Algorithm for Feedback and Feedforward Policies towards Robot Control Robust to Sensing Failures](https://mlsanity.com/paper/2104.00385/) (2021)

## Abstract (arXiv)

This paper proposes a new optimizer for deep learning, named d-AmsGrad. In the real-world data, noise and outliers cannot be excluded from dataset to be used for learning robot skills. This problem is especially striking for robots that learn by collecting data in real time, which cannot be sorted manually. Several noise-robust optimizers have therefore been developed to resolve this problem, and one of them, named AmsGrad, which is a variant of Adam optimizer, has a proof of its convergence. However, in practice, it does not improve learning performance in robotics scenarios. This reason is hypothesized that most of robot learning problems are non-stationary, but AmsGrad assumes the maximum second momentum during learning to be stationarily given. In order to adapt to the non-stationary problems, an improved version, which slowly decays the maximum second momentum, is proposed. The proposed optimizer has the same capability of reaching the global optimum as baselines, and its performance outperformed that of the baselines in robotics problems.