# MAEA: Multimodal Attribution for Embodied AI

arXiv: https://arxiv.org/abs/2307.13850 · HTML: https://mlsanity.com/paper/2307.13850/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 29 references that are in the map.
- First posted on arXiv 25 Jul 2023.

## Builds on (most cited first, 24 of 24)

- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [SmoothGrad: removing noise by adding noise](https://mlsanity.com/paper/1706.03825/) (2017)
- [ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Tasks](https://mlsanity.com/paper/1912.01734/) (2019)
- [Object Goal Navigation using Goal-Oriented Semantic Exploration](https://mlsanity.com/paper/2007.00643/) (2020)
- [Learning to Explore using Active Neural SLAM](https://mlsanity.com/paper/2004.05155/) (2020)
- [Not Just a Black Box: Learning Important Features Through Propagating Activation Differences](https://mlsanity.com/paper/1605.01713/) (2016)
- [TEACh: Task-driven Embodied Agents that Chat](https://mlsanity.com/paper/2110.00534/) (2021)
- [Episodic Transformer for Vision-and-Language Navigation](https://mlsanity.com/paper/2105.06453/) (2021)
- [Visual Semantic Navigation using Scene Priors](https://mlsanity.com/paper/1810.06543/) (2018)
- [FILM: Following Instructions in Language with Modular Methods](https://mlsanity.com/paper/2110.07342/) (2021)
- [Vision-and-Language Navigation: A Survey of Tasks, Methods, and Future Directions](https://mlsanity.com/paper/2203.12667/) (2022)
- [XRAI: Better Attributions Through Regions](https://mlsanity.com/paper/1906.02825/) (2019)
- [Guided Integrated Gradients: An Adaptive Path Method for Removing Noise](https://mlsanity.com/paper/2106.09788/) (2021)
- [Attribution in Scale and Space](https://mlsanity.com/paper/2004.03383/) (2020)
- [Neural Modular Control for Embodied Question Answering](https://mlsanity.com/paper/1810.11181/) (2018)
- [Towards better understanding of gradient-based attribution methods for Deep Neural Networks](https://mlsanity.com/paper/1711.06104/) (2017)
- [Hierarchical Task Learning from Language Instructions with Unified Transformers and Self-Monitoring](https://mlsanity.com/paper/2106.03427/) (2021)
- [Coupling Vision and Proprioception for Navigation of Legged Robots](https://mlsanity.com/paper/2112.02094/) (2021)
- [Factorizing Perception and Policy for Interactive Instruction Following](https://mlsanity.com/paper/2012.03208/) (2020)
- [End-to-end grasping policies for human-in-the-loop robots via deep reinforcement learning](https://mlsanity.com/paper/2104.12842/) (2021)
- [Hierarchical Latent Semantic Mapping for Automated Topic Generation](https://mlsanity.com/paper/1511.03546/) (2015)

## Built on by (most cited first, 9 of 9)

- [CountQA: How Well Do MLLMs Count in the Wild?](https://mlsanity.com/paper/2508.06585/) (2025)
- [GeoChain: Multimodal Chain-of-Thought for Geographic Reasoning](https://mlsanity.com/paper/2506.00785/) (2025)
- [Cutting Through the Noise: Boosting LLM Performance on Math Word Problems](https://mlsanity.com/paper/2406.15444/) (2024)
- [HueManity: Probing Fine-Grained Visual Perception in MLLMs](https://mlsanity.com/paper/2506.03194/) (2025)
- [Semantic Augmentation in Images using Language](https://mlsanity.com/paper/2404.02353/) (2024)
- [Attribution Regularization for Multimodal Paradigms](https://mlsanity.com/paper/2404.02359/) (2024)
- [MaRVL-QA: A Benchmark for Mathematical Reasoning over Visual Landscapes](https://mlsanity.com/paper/2508.17180/) (2025)
- [The Artificial Intelligence Cognitive Examination: A Survey on the Evolution of Multimodal Evaluation from Recognition to Reasoning](https://mlsanity.com/paper/2510.04141/) (2025)
- [Explicit Reasoning Makes Better Judges: A Systematic Study on Accuracy, Efficiency, and Robustness](https://mlsanity.com/paper/2509.13332/) (2025)

## Abstract (arXiv)

Understanding multimodal perception for embodied AI is an open question because such inputs may contain highly complementary as well as redundant information for the task. A relevant direction for multimodal policies is understanding the global trends of each modality at the fusion layer. To this end, we disentangle the attributions for visual, language, and previous action inputs across different policies trained on the ALFRED dataset. Attribution analysis can be utilized to rank and group the failure scenarios, investigate modeling and dataset biases, and critically analyze multimodal EAI policies for robustness and user trust before deployment. We present MAEA, a framework to compute global attributions per modality of any differentiable policy. In addition, we show how attributions enable lower-level behavior analysis in EAI policies for language and visual attributions.