# A Twin Delayed Deep Deterministic Policy Gradient Algorithm for Autonomous Ground Vehicle Navigation via Digital Twin Perception Awareness

arXiv: https://arxiv.org/abs/2403.15067 · HTML: https://mlsanity.com/paper/2403.15067/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in RL for robot navigation and UAV control, one of 15 directions in Deep reinforcement learning methods.
- Builds on 14 of its 39 references that are in the map.
- First posted on arXiv 22 Mar 2024.

## Placed in

- Deep reinforcement learning methods / [RL for robot navigation and UAV control](https://mlsanity.com/deeprl/rl-for-robot-navigation-and-uav-control/)

## Builds on (most cited first, 14 of 14)

- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehicles](https://mlsanity.com/paper/1705.05065/) (2017)
- [Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Survey](https://mlsanity.com/paper/2009.13303/) (2020)
- [LGSVL Simulator: A High Fidelity Simulator for Autonomous Driving](https://mlsanity.com/paper/2005.03778/) (2020)
- [How Simulation Helps Autonomous Driving:A Survey of Sim2real, Digital Twins, and Parallel Intelligence](https://mlsanity.com/paper/2305.01263/) (2023)
- [KITTI-CARLA: a KITTI-like dataset generated by CARLA Simulator](https://mlsanity.com/paper/2109.00892/) (2021)
- [Generalization through Simulation: Integrating Simulated and Real Data into Deep Reinforcement Learning for Vision-Based Autonomous Flight](https://mlsanity.com/paper/1902.03701/) (2019)
- [A Survey on Simulators for Testing Self-Driving Cars](https://mlsanity.com/paper/2101.05337/) (2021)
- [Reinforcement Learning from Simulation to Real World Autonomous Driving using Digital Twin](https://mlsanity.com/paper/2211.14874/) (2022)
- [Comparative Analysis of Control Barrier Functions and Artificial Potential Fields for Obstacle Avoidance](https://mlsanity.com/paper/2010.09819/) (2020)
- [Sequential Neural Barriers for Scalable Dynamic Obstacle Avoidance](https://mlsanity.com/paper/2307.03015/) (2023)
- [Neural Graph Control Barrier Functions Guided Distributed Collision-avoidance Multi-agent Control](https://mlsanity.com/paper/2311.13014/) (2023)
- [Context-Aware Deep Q-Network for Decentralized Cooperative Reconnaissance by a Robotic Swarm](https://mlsanity.com/paper/2001.11710/) (2020)

## Built on by (most cited first, 2 of 2)

- [A Digital Twin Framework for Reinforcement Learning with Real-Time Self-Improvement via Human Assistive Teleoperation](https://mlsanity.com/paper/2406.00732/) (2024)
- [Optimisation of Resource Allocation in Heterogeneous Wireless Networks Using Deep Reinforcement Learning](https://mlsanity.com/paper/2509.25284/) (2025)

## Abstract (arXiv)

Autonomous ground vehicle (UGV) navigation has the potential to revolutionize the transportation system by increasing accessibility to disabled people, ensure safety and convenience of use. However, UGV requires extensive and efficient testing and evaluation to ensure its acceptance for public use. This testing are mostly done in a simulator which result to sim2real transfer gap. In this paper, we propose a digital twin perception awareness approach for the control of robot navigation without prior creation of the virtual environment (VT) environment state. To achieve this, we develop a twin delayed deep deterministic policy gradient (TD3) algorithm that ensures collision avoidance and goal-based path planning. We demonstrate the performance of our approach on different environment dynamics. We show that our approach is capable of efficiently avoiding collision with obstacles and navigating to its desired destination, while at the same time safely avoids obstacles using the information received from the LIDAR sensor mounted on the robot. Our approach bridges the gap between sim-to-real transfer and contributes to the adoption of UGVs in real world. We validate our approach in simulation and a real-world application in an office space.