# Generalization in Deep Reinforcement Learning for Robotic Navigation by Reward Shaping

arXiv: https://arxiv.org/abs/2209.14271 · HTML: https://mlsanity.com/paper/2209.14271/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 54 references that are in the map.
- First posted on arXiv 28 Sep 2022.

## Builds on (most cited first, 18 of 18)

- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [Sim-to-Real Transfer of Robotic Control with Dynamics Randomization](https://mlsanity.com/paper/1710.06537/) (2017)
- [Quantifying Generalization in Reinforcement Learning](https://mlsanity.com/paper/1812.02341/) (2018)
- [Sim-to-Real via Sim-to-Sim: Data-efficient Robotic Grasping via Randomized-to-Canonical Adaptation Networks](https://mlsanity.com/paper/1812.07252/) (2018)
- [A Survey of Zero-shot Generalisation in Deep Reinforcement Learning](https://mlsanity.com/paper/2111.09794/) (2021)
- [Exploration in Deep Reinforcement Learning: A Survey](https://mlsanity.com/paper/2205.00824/) (2022)
- [Learning to Utilize Shaping Rewards: A New Approach of Reward Shaping](https://mlsanity.com/paper/2011.02669/) (2020)
- [Virtual-to-real Deep Reinforcement Learning: Continuous Control of Mobile Robots for Mapless Navigation](https://mlsanity.com/paper/1703.00420/) (2017)
- [Improving Generalization in Reinforcement Learning with Mixture Regularization](https://mlsanity.com/paper/2010.10814/) (2020)
- [Self-supervised Deep Reinforcement Learning with Generalized Computation Graphs for Robot Navigation](https://mlsanity.com/paper/1709.10489/) (2017)
- [Goal-Driven Autonomous Exploration Through Deep Reinforcement Learning](https://mlsanity.com/paper/2103.07119/) (2021)
- [Measuring and Characterizing Generalization in Deep Reinforcement Learning](https://mlsanity.com/paper/1812.02868/) (2018)
- [Deep Reinforcement learning for real autonomous mobile robot navigation in indoor environments](https://mlsanity.com/paper/2005.13857/) (2020)
- [Deep Reinforcement Learning for Mapless Navigation of a Hybrid Aerial Underwater Vehicle with Medium Transition](https://mlsanity.com/paper/2103.12883/) (2021)
- [Sensor Fusion for Robot Control through Deep Reinforcement Learning](https://mlsanity.com/paper/1703.04550/) (2017)
- [Robust Biped Locomotion Using Deep Reinforcement Learning on Top of an Analytical Control Approach](https://mlsanity.com/paper/2104.10592/) (2021)

## Built on by (most cited first, 10 of 10)

- [EDEN: Entorhinal Driven Egocentric Navigation Toward Robotic Deployment](https://mlsanity.com/paper/2506.03046/) (2025)
- [A Zero-Shot Reinforcement Learning Strategy for Autonomous Guidewire Navigation](https://mlsanity.com/paper/2403.02777/) (2024)
- [Long-distance Geomagnetic Navigation in GNSS-denied Environments with Deep Reinforcement Learning](https://mlsanity.com/paper/2410.15837/) (2024)
- [Towards Safe Maneuvering of Double-Ackermann-Steering Robots with a Soft Actor-Critic Framework](https://mlsanity.com/paper/2510.10332/) (2025)
- [Reinforcement Learning for Control Systems with Time Delays: A Comprehensive Survey](https://mlsanity.com/paper/2602.00399/) (2026)
- [ManeuverNet: A Soft Actor-Critic Framework for Precise Maneuvering of Double-Ackermann-Steering Robots with Optimized Reward Functions](https://mlsanity.com/paper/2602.14726/) (2026)
- [DreamSAC: Learning Hamiltonian World Models via Symmetry Exploration](https://mlsanity.com/paper/2603.07545/) (2026)
- [Beyond Visibility Limits: A DRL-Based Navigation Strategy for Unexpected Obstacles](https://mlsanity.com/paper/2503.01127/) (2025)
- [Hierarchical Deep Deterministic Policy Gradient for Autonomous Maze Navigation of Mobile Robots](https://mlsanity.com/paper/2508.04994/) (2025)
- [ContactRL: Safe Reinforcement Learning based Motion Planning for Contact based Human Robot Collaboration](https://mlsanity.com/paper/2512.03707/) (2025)

## Abstract (arXiv)

In this paper, we study the application of DRL algorithms in the context of local navigation problems, in which a robot moves towards a goal location in unknown and cluttered workspaces equipped only with limited-range exteroceptive sensors, such as LiDAR. Collision avoidance policies based on DRL present some advantages, but they are quite susceptible to local minima, once their capacity to learn suitable actions is limited to the sensor range. Since most robots perform tasks in unstructured environments, it is of great interest to seek generalized local navigation policies capable of avoiding local minima, especially in untrained scenarios. To do so, we propose a novel reward function that incorporates map information gained in the training stage, increasing the agent's capacity to deliberate about the best course of action. Also, we use the SAC algorithm for training our ANN, which shows to be more effective than others in the state-of-the-art literature. A set of sim-to-sim and sim-to-real experiments illustrate that our proposed reward combined with the SAC outperforms the compared methods in terms of local minima and collision avoidance.