mlsanityData to 7 Oct 2026

MAEA: Multimodal Attribution for Embodied AI

arXiv 2307.13850 ↗First posted 25 Jul 2023cs.LG cs.AI cs.CVcited by 9

Not on the map. Why?

Abstract (arXiv)

Understanding multimodal perception for embodied AI is an open question because such inputs may contain highly complementary as well as redundant information for the task. A relevant direction for multimodal policies is understanding the global trends of each modality at the fusion layer. To this end, we disentangle the attributions for visual, language, and previous action inputs across different policies trained on the ALFRED dataset. Attribution analysis can be utilized to rank and group the failure scenarios, investigate modeling and dataset biases, and critically analyze multimodal EAI policies for robustness and user trust before deployment. We present MAEA, a framework to compute global attributions per modality of any differentiable policy. In addition, we show how attributions enable lower-level behavior analysis in EAI policies for language and visual attributions.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 24 of its 29 references that are in the map. First posted on arXiv 25 Jul 2023.

Builds on

24 of 29 references.

References · 24

  1. 2017Axiomatic Attribution for Deep Networkscited by 3,350
  2. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  3. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,348
  4. 2016OpenAI Gymcited by 1,684
  5. 2017SmoothGrad: removing noise by adding noisecited by 1,097
  6. 2019ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Taskscited by 773
  7. 2020Object Goal Navigation using Goal-Oriented Semantic Explorationcited by 421
  8. 2020Learning to Explore using Active Neural SLAMcited by 308
  9. 2016Not Just a Black Box: Learning Important Features Through Propagating Activation Differencescited by 295
  10. 2021TEACh: Task-driven Embodied Agents that Chatcited by 194
  11. 2021Episodic Transformer for Vision-and-Language Navigationcited by 155
  12. 2018Visual Semantic Navigation using Scene Priorscited by 127
12 more
  1. 2021FILM: Following Instructions in Language with Modular Methodscited by 119
  2. 2019XRAI: Better Attributions Through Regionscited by 116
  3. 2022Vision-and-Language Navigation: A Survey of Tasks, Methods, and Future Directionscited by 116
  4. 2021Guided Integrated Gradients: An Adaptive Path Method for Removing Noisecited by 102
  5. 2020Attribution in Scale and Spacecited by 61
  6. 2018Neural Modular Control for Embodied Question Answeringcited by 56
  7. 2021Hierarchical Task Learning from Language Instructions with Unified Transformers and Self-Monitoringcited by 53
  8. 2017Towards better understanding of gradient-based attribution methods for Deep Neural Networkscited by 51
  9. 2021Coupling Vision and Proprioception for Navigation of Legged Robotscited by 46
  10. 2020Factorizing Perception and Policy for Interactive Instruction Followingcited by 28
  11. 2021End-to-end grasping policies for human-in-the-loop robots via deep reinforcement learningcited by 2
  12. 2015Hierarchical Latent Semantic Mapping for Automated Topic Generationcited by 1

Built on by

Cited by year

Citing papers per year: 2024: 3; 2025: 6.

9 papers.

  1. 2025CountQA: How Well Do MLLMs Count in the Wild?cited by 17
  2. 2025GeoChain: Multimodal Chain-of-Thought for Geographic Reasoningcited by 16
  3. 2024Cutting Through the Noise: Boosting LLM Performance on Math Word Problemscited by 14
  4. 2025HueManity: Probing Fine-Grained Visual Perception in MLLMscited by 9
  5. 2024Semantic Augmentation in Images using Languagecited by 7
  6. 2024Attribution Regularization for Multimodal Paradigmscited by 7
  7. 2025MaRVL-QA: A Benchmark for Mathematical Reasoning over Visual Landscapescited by 4
  8. 2025The Artificial Intelligence Cognitive Examination: A Survey on the Evolution of Multimodal Evaluation from Recognition to Reasoningcited by 4
  9. 2025Explicit Reasoning Makes Better Judges: A Systematic Study on Accuracy, Efficiency, and Robustnesscited by 3