# Multimodal Aggregation Approach for Memory Vision-Voice Indoor Navigation with Meta-Learning

arXiv: https://arxiv.org/abs/2009.00402 · HTML: https://mlsanity.com/paper/2009.00402/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Sep 2020.

## Built on by (most cited first, 6 of 6)

- [Multimodal Co-learning: Challenges, Applications with Datasets, Recent Advances and Future Directions](https://mlsanity.com/paper/2107.13782/) (2021)
- [Multimodality in Meta-Learning: A Comprehensive Survey](https://mlsanity.com/paper/2109.13576/) (2021)
- [Deep Learning for Embodied Vision Navigation: A Survey](https://mlsanity.com/paper/2108.04097/) (2021)
- [Adaptive Adversarial Training for Meta Reinforcement Learning](https://mlsanity.com/paper/2104.13302/) (2021)
- [Radiance Field Learners As UAV First-Person Viewers](https://mlsanity.com/paper/2408.05533/) (2024)
- [RL3: Boosting Meta Reinforcement Learning via RL inside RL2](https://mlsanity.com/paper/2306.15909/) (2023)

## Abstract (arXiv)

Vision and voice are two vital keys for agents' interaction and learning. In this paper, we present a novel indoor navigation model called Memory Vision-Voice Indoor Navigation (MVV-IN), which receives voice commands and analyzes multimodal information of visual observation in order to enhance robots' environment understanding. We make use of single RGB images taken by a first-view monocular camera. We also apply a self-attention mechanism to keep the agent focusing on key areas. Memory is important for the agent to avoid repeating certain tasks unnecessarily and in order for it to adapt adequately to new scenes, therefore, we make use of meta-learning. We have experimented with various functional features extracted from visual observation. Comparative experiments prove that our methods outperform state-of-the-art baselines.