# GeoRisk-RAG: A Hierarchy-Aware Risk Framework for Improving RAG Reliability through Selective Answering

arXiv: https://arxiv.org/abs/2608.22634 · HTML: https://mlsanity.com/paper/2608.22634/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RAG for recommendation, one of 15 directions in Retrieval-augmented generation.
- Builds on 13 of its 43 references that are in the map.
- First posted on arXiv 23 Aug 2026.

## Placed in

- Retrieval-augmented generation / [RAG for recommendation](https://mlsanity.com/rag/rag-for-recommendation/)

## Builds on (most cited first, 13 of 13)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers](https://mlsanity.com/paper/2002.10957/) (2020)
- [DisasterResponseGPT: Large Language Models for Accelerated Plan of Action Development in Disaster Response Scenarios](https://mlsanity.com/paper/2306.17271/) (2023)
- [WildfireGPT: Tailored Large Language Model for Wildfire Analysis](https://mlsanity.com/paper/2402.07877/) (2024)
- [Knowledge Graph-extended Retrieval Augmented Generation for Question Answering](https://mlsanity.com/paper/2504.08893/) (2025)
- [A Comparative Study of PDF Parsing Tools Across Diverse Document Categories](https://mlsanity.com/paper/2410.09871/) (2024)
- [Shallow Cross-Encoders for Low-Latency Retrieval](https://mlsanity.com/paper/2403.20222/) (2024)
- [LLM4VV: Exploring LLM-as-a-Judge for Validation and Verification Testsuites](https://mlsanity.com/paper/2408.11729/) (2024)
- [NER4all or Context is All You Need: Using LLMs for low-effort, high-performance NER on historical texts. A humanities informed approach](https://mlsanity.com/paper/2502.04351/) (2025)
- [GeoRAG: A Question-Answering Approach from a Geographical Perspective](https://mlsanity.com/paper/2504.01458/) (2025)
- [Geolocating News about Extreme Climate Events: A Comparative Analysis of Off-the-Shelf Tools for Toponym Identification in German](https://mlsanity.com/paper/2605.03414/) (2026)
- [SafeMate: A Modular RAG-Based Agent for Context-Aware Emergency Guidance](https://mlsanity.com/paper/2505.02306/) (2025)
- [DisasterVQA: A Visual Question Answering Benchmark Dataset for Disaster Scenes](https://mlsanity.com/paper/2601.13839/) (2026)

## Abstract (arXiv)

Current work on improving reliability in large language model (LLM)- generated answers has primarily leveraged Retrieval-Augmented Generation (RAG), knowledge-graph augmentation, and reinforcement learning. While these methods are adept at enhancing and measuring reliability through semantic similarity and faithfulness, they often struggle to distinguish semantic similarity from geographic validity. This is especially critical in natural hazard management domains where geographic granularity (i.e., town vs. city vs. state) is significant for decision-making, as responses valid in one municipality may not transfer to another. In such domains, a confidently wrong answer carries greater risk than abstaining. We present GeoRisk-RAG, a novel hierarchy-aware framework that addresses this geographic-validity gap through selective answering. This framework explicitly estimates geographic applicability using a Directed Acyclic Graph (DAG)-based distance for context retrieval before response generation. Experiments on a novel held-out wildfire-related question-answering (QA) dataset show that GeoRisk-RAG significantly reduces false confidence rates for location-dependent questions, lowering the rate to 0.009 compared with ~0.090 for standard semantic similarity and reranking baselines, while consistently achieving higher human preference alignment. This work provides a more comprehensive assessment of end-to-end RAG pipelines by integrating geographic validity and selective-answering behavior for safer decision-making in geospatial domains.