# A Self Consistency Based Reranking for Narrative Question Answering

arXiv: https://arxiv.org/abs/2606.15741 · HTML: https://mlsanity.com/paper/2606.15741/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning.
- Builds on 25 of its 30 references that are in the map.
- First posted on arXiv 14 Jun 2026.

## Placed in

- LLM reasoning / [Test-time compute and efficient thinking](https://mlsanity.com/lr/test-time-compute-and-efficient-thinking/)

## Builds on (most cited first, 25 of 25)

- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Let's Verify Step by Step](https://mlsanity.com/paper/2305.20050/) (2023)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [Universal Self-Consistency for Large Language Model Generation](https://mlsanity.com/paper/2311.17311/) (2023)
- [Reasoning Aware Self-Consistency: Leveraging Reasoning Paths for Efficient LLM Sampling](https://mlsanity.com/paper/2408.17017/) (2024)
- [Better patching using LLM prompting, via Self-Consistency](https://mlsanity.com/paper/2306.00108/) (2023)
- [Large Language Model (LLM) AI text generation detection based on transformer deep learning algorithm](https://mlsanity.com/paper/2405.06652/) (2024)
- [Importance of Self-Consistency in Active Learning for Semantic Segmentation](https://mlsanity.com/paper/2008.01860/) (2020)
- [MedSyn: LLM-based Synthetic Medical Text Generation Framework](https://mlsanity.com/paper/2408.02056/) (2024)
- [Bridging Internal Probability and Self-Consistency for Effective and Efficient LLM Reasoning](https://mlsanity.com/paper/2502.00511/) (2025)
- [A Theoretical Study on Bridging Internal Probability and Self-Consistency for LLM Reasoning](https://mlsanity.com/paper/2510.15444/) (2025)
- [Revisiting Self-Consistency from Dynamic Distributional Alignment Perspective on Answer Aggregation](https://mlsanity.com/paper/2502.19830/) (2025)
- [Self-Critique and Refinement for Faithful Natural Language Explanations](https://mlsanity.com/paper/2505.22823/) (2025)
- [LLM-DaaS: LLM-driven Drone-as-a-Service Operations from Text User Requests](https://mlsanity.com/paper/2412.11672/) (2024)
- [Self-Improvement of Language Models by Post-Training on Multi-Agent Debate](https://mlsanity.com/paper/2509.15172/) (2025)
- [LLM-SEM: A Sentiment-Based Student Engagement Metric Using LLMS for E-Learning Platforms](https://mlsanity.com/paper/2412.13765/) (2024)
- [RIRO: Reshaping Inputs, Refining Outputs Unlocking the Potential of Large Language Models in Data-Scarce Contexts](https://mlsanity.com/paper/2412.15254/) (2024)
- [ERPA: Efficient RPA Model Integrating OCR and LLMs for Intelligent Document Processing](https://mlsanity.com/paper/2412.19840/) (2024)
- [Beyond Self-Consistency: Ensemble Reasoning Boosts Consistency and Accuracy of LLMs in Cancer Staging](https://mlsanity.com/paper/2404.13149/) (2024)
- [An Ensemble Classification Approach in A Multi-Layered Large Language Model Framework for Disease Prediction](https://mlsanity.com/paper/2509.02446/) (2025)
- [Scalable Consistency Training for Graph Neural Networks via Self-Ensemble Self-Distillation](https://mlsanity.com/paper/2110.06290/) (2021)
- [Two-Stage Quranic QA via Ensemble Retrieval and Instruction-Tuned Answer Extraction](https://mlsanity.com/paper/2508.06971/) (2025)
- [Severity-Aware Weighted Loss for Arabic Medical Text Generation](https://mlsanity.com/paper/2604.06346/) (2026)
- [A Severity-Based Curriculum Learning Strategy for Arabic Medical Text Generation](https://mlsanity.com/paper/2604.06365/) (2026)
- [Confidence-Credibility Aware Weighted Ensembles of Small LLMs Outperform Large LLMs in Emotion Detection](https://mlsanity.com/paper/2512.17630/) (2025)

## Abstract (arXiv)

Narrative question answering (NQA) is a challenging task in natural language processing that requires models to understand long textual contexts, capture relationships across events, and generate coherent responses. Despite recent advances in pretrained language models, most existing approaches rely on a single decoding output during inference, making them sensitive to generation variability and often resulting in incomplete or inconsistent answers .To address this limitation, we propose a self-ensemble Self-Consistency-Based reranking framework for narrative question answering. The proposed method generates multiple candidate answers for each story-question pair and selects the final answer based on semantic agreement among the generated responses. This allows the model to explore diverse answer formulations while improving robustness through consensus-based selection without requiring modifications to the underlying architecture .The framework combines pretrained and fine-tuned language generation with multi-answer inference and similarity-based reranking. We evaluate the proposed approach on the NarrativeQA dataset using multiple models, including FLAN-T5 (Base and Small) and Pegasus-Large, under both baseline and fine-tuned settings .Experimental results demonstrate that the proposed method consistently improves performance across all models. In particular, FLAN-T5-Base achieves the best overall performance, improving from 82.32% to 86.66% (+4.34%) when combined with self-ensemble inference. Additionally, the largest improvement is observed with Pegasus-Large, which increases from 72.50% to 87.07% (+14.57%), highlighting the effectiveness of the proposed strategy.