# Socially-Aware Robot Navigation Enhanced by Bidirectional Natural Language Conversations Using Large Language Models

arXiv: https://arxiv.org/abs/2409.04965 · HTML: https://mlsanity.com/paper/2409.04965/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Sep 2024.

## Built on by (most cited first, 3 of 3)

- [Driving with Regulation: Trustworthy and Interpretable Decision-Making for Autonomous Driving with Retrieval-Augmented Reasoning](https://mlsanity.com/paper/2410.04759/) (2024)
- [Humanoid Agent via Embodied Chain-of-Action Reasoning with Multimodal Foundation Models for Zero-Shot Loco-Manipulation](https://mlsanity.com/paper/2504.09532/) (2025)
- [When Robots Mishear Us: Mapping the Safety Risks of Voice-Controlled Embodied AI](https://mlsanity.com/paper/2608.28518/) (2026)

## Abstract (arXiv)

Robot navigation is crucial across various domains, yet traditional methods focus on efficiency and obstacle avoidance, often overlooking human behavior in shared spaces. With the rise of service robots, socially aware navigation has gained prominence. However, existing approaches primarily predict pedestrian movements or issue alerts, lacking true human-robot interaction. We introduce Hybrid Soft Actor-Critic with Large Language Model (HSAC-LLM), a novel framework for socially aware navigation. By integrating deep reinforcement learning with large language models, HSAC-LLM enables bidirectional natural language interactions, predicting both continuous and discrete navigation actions. When potential collisions arise, the robot proactively communicates with pedestrians to determine avoidance strategies. Experiments in 2D simulation, Gazebo, and real-world environments demonstrate that HSAC-LLM outperforms state-of-the-art DRL methods in interaction, navigation, and obstacle avoidance. This paradigm advances effective human-robot interactions in dynamic settings. Videos are available at https://hsacllm.github.io/.