# Towards Language-Based Modulation of Assistive Robots through Multimodal Models

arXiv: https://arxiv.org/abs/2306.14830 · HTML: https://mlsanity.com/paper/2306.14830/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Jun 2023.

## Built on by (most cited first, 1 of 1)

- [Probing Language Models' Gesture Understanding for Enhanced Human-AI Interaction](https://mlsanity.com/paper/2401.17858/) (2024)

## Abstract (arXiv)

In the field of Geriatronics, enabling effective and transparent communication between humans and robots is crucial for enhancing the acceptance and performance of assistive robots. Our early-stage research project investigates the potential of language-based modulation as a means to improve human-robot interaction. We propose to explore real-time modulation during task execution, leveraging language cues, visual references, and multimodal inputs. By developing transparent and interpretable methods, we aim to enable robots to adapt and respond to language commands, enhancing their usability and flexibility. Through the exchange of insights and knowledge at the workshop, we seek to gather valuable feedback to advance our research and contribute to the development of interactive robotic systems for Geriatronics and beyond.