# On the Importance of Karaka Framework in Multi-modal Grounding

arXiv: https://arxiv.org/abs/2204.04347 · HTML: https://mlsanity.com/paper/2204.04347/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 28 references that are in the map.
- First posted on arXiv 9 Apr 2022.

## Builds on (most cited first, 11 of 11)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [VQA: Visual Question Answering](https://mlsanity.com/paper/1505.00468/) (2015)
- [ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Tasks](https://mlsanity.com/paper/1912.01734/) (2019)
- [Speaker-Follower Models for Vision-and-Language Navigation](https://mlsanity.com/paper/1806.02724/) (2018)
- [Episodic Transformer for Vision-and-Language Navigation](https://mlsanity.com/paper/2105.06453/) (2021)
- [Diagnosing the Environment Bias in Vision-and-Language Navigation](https://mlsanity.com/paper/2005.03086/) (2020)
- [Diagnosing Vision-and-Language Navigation: What Really Matters](https://mlsanity.com/paper/2103.16561/) (2021)
- [Natural Language Interaction with Explainable AI Models](https://mlsanity.com/paper/1903.05720/) (2019)
- [X-ToM: Explaining with Theory-of-Mind for Gaining Justified Human Trust](https://mlsanity.com/paper/1909.06907/) (2019)
- [Effective Representation to Capture Collaboration Behaviors between Explainer and User](https://mlsanity.com/paper/2201.03147/) (2022)
- [Question Generation for Evaluating Cross-Dataset Shifts in Multi-modal Grounding](https://mlsanity.com/paper/2201.09639/) (2022)

## Abstract (arXiv)

Computational Paninian Grammar model helps in decoding a natural language expression as a series of modifier-modified relations and therefore facilitates in identifying dependency relations closer to language (context) semantics compared to the usual Stanford dependency relations. However, the importance of this CPG dependency scheme has not been studied in the context of multi-modal vision and language applications. At IIIT Hyderabad, we plan to perform a novel study to explore the potential advantages and disadvantages of CPG framework in a vision-language navigation task setting, a popular and challenging multi-modal grounding task.