# Multi-modal Feedback for Affordance-driven Interactive Reinforcement Learning

arXiv: https://arxiv.org/abs/1807.09991 · HTML: https://mlsanity.com/paper/1807.09991/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Jul 2018.

## Built on by (most cited first, 9 of 9)

- [Survey on reinforcement learning for language processing](https://mlsanity.com/paper/2104.05565/) (2021)
- [Temporally Abstract Partial Models](https://mlsanity.com/paper/2108.03213/) (2021)
- [Recent Advances of Deep Robotic Affordance Learning: A Reinforcement Learning Perspective](https://mlsanity.com/paper/2303.05344/) (2023)
- [Persistent Rule-based Interactive Reinforcement Learning](https://mlsanity.com/paper/2102.02441/) (2021)
- [The Paradox of Choice: Using Attention in Hierarchical Reinforcement Learning](https://mlsanity.com/paper/2201.09653/) (2022)
- [GrASP: Gradient-Based Affordance Selection for Planning](https://mlsanity.com/paper/2202.04772/) (2022)
- [On the Effect of Robot Errors on Human Teaching Dynamics](https://mlsanity.com/paper/2409.09827/) (2024)
- [Prosody as a Teaching Signal for Agent Learning: Exploratory Studies and Algorithmic Implications](https://mlsanity.com/paper/2410.23554/) (2024)
- [NatSGLD: A Dataset with Speech, Gesture, Logic, and Demonstration for Robot Learning in Natural Human-Robot Interaction](https://mlsanity.com/paper/2502.16718/) (2025)

## Abstract (arXiv)

Interactive reinforcement learning (IRL) extends traditional reinforcement learning (RL) by allowing an agent to interact with parent-like trainers during a task. In this paper, we present an IRL approach using dynamic audio-visual input in terms of vocal commands and hand gestures as feedback. Our architecture integrates multi-modal information to provide robust commands from multiple sensory cues along with a confidence value indicating the trustworthiness of the feedback. The integration process also considers the case in which the two modalities convey incongruent information. Additionally, we modulate the influence of sensory-driven feedback in the IRL task using goal-oriented knowledge in terms of contextual affordances. We implement a neural network architecture to predict the effect of performed actions with different objects to avoid failed-states, i.e., states from which it is not possible to accomplish the task. In our experimental setup, we explore the interplay of multimodal feedback and task-specific affordances in a robot cleaning scenario. We compare the learning performance of the agent under four different conditions: traditional RL, multi-modal IRL, and each of these two setups with the use of contextual affordances. Our experiments show that the best performance is obtained by using audio-visual feedback with affordancemodulated IRL. The obtained results demonstrate the importance of multi-modal sensory processing integrated with goal-oriented knowledge in IRL tasks.