Long-context Non-factoid Question Answering in Indic Languages
Not on the map. Why?
Abstract (arXiv)
Question Answering (QA) tasks, which involve extracting answers from a given context, are relatively straightforward for modern Large Language Models (LLMs) when the context is short. However, long contexts pose challenges due to the quadratic complexity of the self-attention mechanism. This challenge is compounded in Indic languages, which are often low-resource. This study explores context-shortening techniques, including Open Information Extraction (OIE), coreference resolution, Answer Paragraph Selection (APS), and their combinations, to improve QA performance. Compared to the baseline of unshortened (long) contexts, our experiments on four Indic languages (Hindi, Tamil, Telugu, and Urdu) demonstrate that context-shortening techniques yield an average improvement of 4% in semantic scores and 47% in token-level scores when evaluated on three popular LLMs without fine-tuning. Furthermore, with fine-tuning, we achieve an average increase of 2% in both semantic and token-level scores. Additionally, context-shortening reduces computational overhead. Explainability techniques like LIME and SHAP reveal that when the APS model confidently identifies the paragraph containing the answer, nearly all tokens within the selected text receive high relevance scores. However, the study also highlights the limitations of LLM-based QA systems in addressing non-factoid questions, particularly those requiring reasoning or debate. Moreover, verbalizing OIE-generated triples does not enhance system performance. These findings emphasize the potential of context-shortening techniques to improve the efficiency and effectiveness of LLM-based QA systems, especially for low-resource languages. The source code and resources are available at https://github.com/ritwikmishra/IndicGenQA.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 42 of its 91 references that are in the map. First posted on arXiv 18 Apr 2025.
Builds on
42 of 91 references.
References · 42
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
- 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
- 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
- 2017A Unified Approach to Interpreting Model Predictionscited by 6,555
- 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
- 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,070
- 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
- 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
- 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
- 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,417
- 2020XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalizationcited by 742
30 more
- 2020Long Range Arena: A Benchmark for Efficient Transformerscited by 700
- 2018Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyondcited by 434
- 2024∞Bench: Extending Long Context Evaluation Beyond 100K Tokenscited by 367
- 2024Humans or LLMs as the Judge? A Study on Judgement Biasescited by 323
- 2019K-BERT: Enabling Language Representation with Knowledge Graphcited by 304
- 2024Long-context LLMs Struggle with Long In-context Learningcited by 286
- 2023Knowledge-Augmented Language Model Prompting for Zero-Shot Knowledge Graph Question Answeringcited by 119
- 2022Re2G: Retrieve, Rerank, Generatecited by 102
- 2024Towards Building Multilingual Language Model for Medicinecited by 92
- 2019TANDA: Transfer and Adapt Pre-Trained Transformer Models for Answer Sentence Selectioncited by 74
- 2017Learning Joint Multilingual Sentence Representations with Neural Machine Translationcited by 52
- 2019Temporal Reasoning via Audio Question Answeringcited by 39
- 2024Human-Centered Design Recommendations for LLM-as-a-Judgecited by 37
- 2017Making Neural QA as Simple as Possible but not Simplercited by 33
- 2023Retrieving Supporting Evidence for Generative Question Answeringcited by 31
- 2024Preference Tuning For Toxicity Mitigation Generalizes Across Languagescited by 31
- 2022ferret: a Framework for Benchmarking Explainers on Transformerscited by 26
- 2017Answering Complex Questions Using Open Information Extractioncited by 25
- 2017Learning by Asking Questionscited by 25
- 2015Applying Deep Learning to Answer Selection: A Study and An Open Taskcited by 24
- 2019ANTIQUE: A Non-Factoid Question Answering Benchmarkcited by 24
- 2024INDIC QA BENCHMARK: A Multilingual Benchmark to Evaluate Question Answering capability of LLMs for Indic Languagescited by 24
- 2018WikiPassageQA: A Benchmark Collection for Research on Non-factoid Answer Passage Retrievalcited by 19
- 2021Long Context Question Answering via Supervised Contrastive Learningcited by 17
- 2024Bridging Context Gaps: Leveraging Coreference Resolution for Long Contextual Understandingcited by 16
- 2024MM-PhyQA: Multimodal Physics Question-Answering With Multi-Image CoT Promptingcited by 15
- 2016Question Answering via Integer Programming over Semi-Structured Knowledgecited by 13
- 2024Improving Multimodal LLMs Ability In Geometry Problem Solving, Reasoning, And Multistep Scoringcited by 5
- 2024Multilingual Mathematical Reasoning: Advancing Open-Source LLMs in Hindi and Englishcited by 5
- 2019Answer Interaction in Non-factoid Question Answering Systemscited by 2
Built on by
Citing papers per year: 2026: 2.
2 papers.