# Visualizing the Loss Landscape of Actor Critic Methods with Applications in Inventory Optimization

arXiv: https://arxiv.org/abs/2009.02391 · HTML: https://mlsanity.com/paper/2009.02391/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Sep 2020.

## Built on by (most cited first, 5 of 5)

- [Policy Optimization in a Noisy Neighborhood: On Return Landscapes in Continuous Control](https://mlsanity.com/paper/2309.14597/) (2023)
- [Cliff Diving: Exploring Reward Surfaces in Reinforcement Learning Environments](https://mlsanity.com/paper/2205.07015/) (2022)
- [Investigating the Impact of Action Representations in Policy Gradient Algorithms](https://mlsanity.com/paper/2309.06921/) (2023)
- [Enhancing Policy Gradient with the Polyak Step-Size Adaption](https://mlsanity.com/paper/2404.07525/) (2024)
- [Visualizing Critic Match Loss Landscapes for Interpretation of Online Reinforcement Learning Control Algorithms](https://mlsanity.com/paper/2603.14535/) (2026)

## Abstract (arXiv)

Continuous control is a widely applicable area of reinforcement learning. The main players of this area are actor-critic methods that utilize policy gradients of neural approximators as a common practice. The focus of our study is to show the characteristics of the actor loss function which is the essential part of the optimization. We exploit low dimensional visualizations of the loss function and provide comparisons for loss landscapes of various algorithms. Furthermore, we apply our approach to multi-store dynamic inventory control, a notoriously difficult problem in supply chain operations, and explore the shape of the loss function associated with the optimal policy. We modelled and solved the problem using reinforcement learning while having a loss landscape in favor of optimality.