# Multi-LLM QA with Embodied Exploration

arXiv: https://arxiv.org/abs/2406.10918 · HTML: https://mlsanity.com/paper/2406.10918/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 45 references that are in the map.
- First posted on arXiv 16 Jun 2024.

## Builds on (most cited first, 26 of 26)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Improving Factuality and Reasoning in Language Models through Multiagent Debate](https://mlsanity.com/paper/2305.14325/) (2023)
- [Matterport3D: Learning from RGB-D Data in Indoor Environments](https://mlsanity.com/paper/1709.06158/) (2017)
- [ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debate](https://mlsanity.com/paper/2308.07201/) (2023)
- [Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AI](https://mlsanity.com/paper/2109.08238/) (2021)
- [LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusion](https://mlsanity.com/paper/2306.02561/) (2023)
- [Mixture-of-Agents Enhances Large Language Model Capabilities](https://mlsanity.com/paper/2406.04692/) (2024)
- [All Models are Wrong, but Many are Useful: Learning a Variable's Importance by Studying an Entire Class of Prediction Models Simultaneously](https://mlsanity.com/paper/1801.01489/) (2018)
- [Embodied Question Answering](https://mlsanity.com/paper/1711.11543/) (2017)
- [ReConcile: Round-Table Conference Improves Reasoning via Consensus among Diverse LLMs](https://mlsanity.com/paper/2309.13007/) (2023)
- [A Survey of Embodied AI: From Simulators to Research Tasks](https://mlsanity.com/paper/2103.04918/) (2021)
- [Habitat 3.0: A Co-Habitat for Humans, Avatars and Robots](https://mlsanity.com/paper/2310.13724/) (2023)
- [Rethinking the Bounds of LLM Reasoning: Are Multi-Agent Discussions the Key?](https://mlsanity.com/paper/2402.18272/) (2024)
- [IQA: Visual Question Answering in Interactive Environments](https://mlsanity.com/paper/1712.03316/) (2017)
- [PRD: Peer Rank and Discussion Improve Large Language Model based Evaluations](https://mlsanity.com/paper/2307.02762/) (2023)
- [CinePile: A Long Video Question Answering Dataset and Benchmark](https://mlsanity.com/paper/2405.08813/) (2024)
- [Embodied Question Answering in Photorealistic Environments with Point Cloud Perception](https://mlsanity.com/paper/1904.03461/) (2019)
- [Examining Inter-Consistency of Large Language Models Collaboration: An In-depth Analysis via Debate](https://mlsanity.com/paper/2305.11595/) (2023)
- [Multi-Target Embodied Question Answering](https://mlsanity.com/paper/1904.04686/) (2019)
- [Getting MoRE out of Mixture of Language Model Reasoning Experts](https://mlsanity.com/paper/2305.14628/) (2023)
- [MetaQA: Combining Expert Agents for Multi-Skill Question Answering](https://mlsanity.com/paper/2112.01922/) (2021)
- [What Should I Do Now? Marrying Reinforcement Learning and Symbolic Planning](https://mlsanity.com/paper/1901.01492/) (2019)
- [SoundSpaces: Audio-Visual Navigation in 3D Environments](https://mlsanity.com/paper/1912.11474/) (2019)
- [Evaluating Zero-Shot GPT-4V Performance on 3D Visual Question Answering Benchmarks](https://mlsanity.com/paper/2405.18831/) (2024)
- [Embodied Referring Expression for Manipulation Question Answering in Interactive Environment](https://mlsanity.com/paper/2210.02709/) (2022)

## Built on by (most cited first, 3 of 3)

- [HANRAG: Heuristic Accurate Noise-resistant Retrieval-Augmented Generation for Multi-hop Question Answering](https://mlsanity.com/paper/2509.09713/) (2025)
- [Scaffolding Foundation Models into Physical-World Agents Pushes the Frontier of Long-Horizon Navigation](https://mlsanity.com/paper/2608.30396/) (2026)
- [Boosting Embodied AI Agents through Perception-Generation Disaggregation and Asynchronous Pipeline Execution](https://mlsanity.com/paper/2509.09560/) (2025)

## Abstract (arXiv)

Large language models (LLMs) have grown in popularity due to their natural language interface and pre trained knowledge, leading to rapidly increasing success in question-answering (QA) tasks. More recently, multi-agent systems with LLM-based agents (Multi-LLM) have been utilized increasingly more for QA. In these scenarios, the models may each answer the question and reach a consensus or each model is specialized to answer different domain questions. However, most prior work dealing with Multi-LLM QA has focused on scenarios where the models are asked in a zero-shot manner or are given information sources to extract the answer. For question answering of an unknown environment, embodied exploration of the environment is first needed to answer the question. This skill is necessary for personalizing embodied AI to environments such as households. There is a lack of insight into whether a Multi-LLM system can handle question-answering based on observations from embodied exploration. In this work, we address this gap by investigating the use of Multi-Embodied LLM Explorers (MELE) for QA in an unknown environment. Multiple LLM-based agents independently explore and then answer queries about a household environment. We analyze different aggregation methods to generate a single, final answer for each query: debating, majority voting, and training a central answer module (CAM). Using CAM, we observe a 46% higher accuracy compared against the other non-learning-based aggregation methods. We provide code and the query dataset for further research.