# MQA: Answering the Question via Robotic Manipulation

arXiv: https://arxiv.org/abs/2003.04641 · HTML: https://mlsanity.com/paper/2003.04641/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Mar 2020.

## Built on by (most cited first, 11 of 11)

- [ARNOLD: A Benchmark for Language-Grounded Task Learning With Continuous States in Realistic 3D Scenes](https://mlsanity.com/paper/2304.04321/) (2023)
- [Survey on reinforcement learning for language processing](https://mlsanity.com/paper/2104.05565/) (2021)
- [A Survey on Efficient Vision-Language Models](https://mlsanity.com/paper/2504.09724/) (2025)
- [Internally Rewarded Reinforcement Learning](https://mlsanity.com/paper/2302.00270/) (2023)
- [Embodied Referring Expression for Manipulation Question Answering in Interactive Environment](https://mlsanity.com/paper/2210.02709/) (2022)
- [LEMMA: Learning Language-Conditioned Multi-Robot Manipulation](https://mlsanity.com/paper/2308.00937/) (2023)
- [Heterogeneous Embodied Multi-Agent Collaboration](https://mlsanity.com/paper/2307.13957/) (2023)
- [SpatiaLQA: A Benchmark for Evaluating Spatial Logical Reasoning in Vision-Language Models](https://mlsanity.com/paper/2602.20901/) (2026)
- [Deep Learning for Embodied Vision Navigation: A Survey](https://mlsanity.com/paper/2108.04097/) (2021)
- [PROBE: Manipulation-Grounded Visual Question Answering with VLM Agents](https://mlsanity.com/paper/2608.17129/) (2026)
- [iLearnRobot: An Interactive Learning-Based Multi-Modal Robot with Continuous Improvement](https://mlsanity.com/paper/2507.22896/) (2025)

## Abstract (arXiv)

In this paper, we propose a novel task, Manipulation Question Answering (MQA), where the robot performs manipulation actions to change the environment in order to answer a given question. To solve this problem, a framework consisting of a QA module and a manipulation module is proposed. For the QA module, we adopt the method for the Visual Question Answering (VQA) task. For the manipulation module, a Deep Q Network (DQN) model is designed to generate manipulation actions for the robot to interact with the environment. We consider the situation where the robot continuously manipulating objects inside a bin until the answer to the question is found. Besides, a novel dataset that contains a variety of object models, scenarios and corresponding question-answer pairs is established in a simulation environment. Extensive experiments have been conducted to validate the effectiveness of the proposed framework.