# Structure-Aware RAG: Structured Retrieval Augmented Generation from Noisy Data for Conversational Agents

arXiv: https://arxiv.org/abs/2605.24366 · HTML: https://mlsanity.com/paper/2605.24366/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Conversational and speech RAG, one of 15 directions in Retrieval-augmented generation.
- Builds on 24 of its 37 references that are in the map.
- First posted on arXiv 23 May 2026.

## Placed in

- Retrieval-augmented generation / [Conversational and speech RAG](https://mlsanity.com/rag/conversational-and-speech-rag/)
- LLM agents and tool use / [Web, search, and research agents](https://mlsanity.com/agents/web-search-and-research-agents/)

## Builds on (most cited first, 24 of 24)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [From Local to Global: A Graph RAG Approach to Query-Focused Summarization](https://mlsanity.com/paper/2404.16130/) (2024)
- [DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation](https://mlsanity.com/paper/1911.00536/) (2019)
- [Towards a Human-like Open-Domain Chatbot](https://mlsanity.com/paper/2001.09977/) (2020)
- [TAPAS: Weakly Supervised Table Parsing via Pre-training](https://mlsanity.com/paper/2004.02349/) (2020)
- [TaBERT: Pretraining for Joint Understanding of Textual and Tabular Data](https://mlsanity.com/paper/2005.08314/) (2020)
- [The Power of Noise: Redefining Retrieval for RAG Systems](https://mlsanity.com/paper/2401.14887/) (2024)
- [TAPEX: Table Pre-training via Learning a Neural SQL Executor](https://mlsanity.com/paper/2107.07653/) (2021)
- [Yara Parser: A Fast and Accurate Dependency Parser](https://mlsanity.com/paper/1503.06733/) (2015)
- [Summary of a Haystack: A Challenge to Long-Context LLMs and RAG Systems](https://mlsanity.com/paper/2407.01370/) (2024)
- [Analyzing and Characterizing User Intent in Information-seeking Conversations](https://mlsanity.com/paper/1804.08759/) (2018)
- [Removal of Hallucination on Hallucination: Debate-Augmented RAG](https://mlsanity.com/paper/2505.18581/) (2025)
- [LLM-Powered AI Agent Systems and Their Applications in Industry](https://mlsanity.com/paper/2505.16120/) (2025)
- [User Intent Prediction in Information-seeking Conversations](https://mlsanity.com/paper/1901.03489/) (2019)
- [Response Ranking with Deep Matching Networks and External Knowledge in Information-seeking Conversation Systems](https://mlsanity.com/paper/1805.00188/) (2018)
- [TARGET: Benchmarking Table Retrieval for Generative Tasks](https://mlsanity.com/paper/2505.11545/) (2025)
- [Enhancing Q&A Text Retrieval with Ranking Models: Benchmarking, fine-tuning and deploying Rerankers for RAG](https://mlsanity.com/paper/2409.07691/) (2024)
- [Enhancing Retrieval and Managing Retrieval: A Four-Module Synergy for Improved Quality and Efficiency in RAG Systems](https://mlsanity.com/paper/2407.10670/) (2024)
- [Reassessing Extractive QA Datasets at Scale: LLM-as-a-Judge and In-Depth Analyses](https://mlsanity.com/paper/2504.11972/) (2025)
- [Knowledge Graph Augmented Large Language Models for Disease Prediction](https://mlsanity.com/paper/2512.01210/) (2025)
- [KG-CQR: Leveraging Structured Relation Representations in Knowledge Graphs for Contextual Query Retrieval](https://mlsanity.com/paper/2508.20417/) (2025)
- [Securing LLM-as-a-Service for Small Businesses: An Industry Case Study of a Distributed Chatbot Deployment Platform](https://mlsanity.com/paper/2601.15528/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) have been widely adopted in conversational applications. However, their reliance on parametric knowledge limits reliability in real-world scenarios that require dynamic or domain-specific information. Retrieval-Augmented Generation (RAG) addresses this limitation by incorporating external knowledge during generation, but existing text-based and graph-based RAG methods often struggle with noisy or irrelevant contexts. In this work, we propose Structure-aware Retrieval Augmented Generation (SA-RAG), which uses tables as an intermediate structured representation to provide a compact and controllable interface that reduces noise while preserving essential information. We introduce a quality-aware table metadata generation framework that models metadata normalization and effectiveness, improving metadata quality and downstream performance. Furthermore, we explore both training-free and training-based table generation methods. Generation validation and direct preference optimization further improve table quality while maintaining semantic and structural consistency. Experiments on two noisy real-world datasets show that SA-RAG significantly outperforms existing RAG baselines. Our code is publicly available at a public repository.