# Arming Data Agents with Tribal Knowledge

arXiv: https://arxiv.org/abs/2602.13521 · HTML: https://mlsanity.com/paper/2602.13521/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use.
- Builds on 26 of its 44 references that are in the map.
- First posted on arXiv 13 Feb 2026.

## Placed in

- LLM agents and tool use / [Web, search, and research agents](https://mlsanity.com/agents/web-search-and-research-agents/)

## Builds on (most cited first, 26 of 26)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023)
- [Toolformer: Language Models Can Teach Themselves to Use Tools](https://mlsanity.com/paper/2302.04761/) (2023)
- [A-MEM: Agentic Memory for LLM Agents](https://mlsanity.com/paper/2502.12110/) (2025)
- [Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLs](https://mlsanity.com/paper/2305.03111/) (2023)
- [Text-to-SQL Empowered by Large Language Models: A Benchmark Evaluation](https://mlsanity.com/paper/2308.15363/) (2023)
- [Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflows](https://mlsanity.com/paper/2411.07763/) (2024)
- [CodeS: Towards Building Open-source Language Models for Text-to-SQL](https://mlsanity.com/paper/2402.16347/) (2024)
- [CHASE-SQL: Multi-Path Reasoning and Preference Optimized Candidate Selection in Text-to-SQL](https://mlsanity.com/paper/2410.01943/) (2024)
- [Next-Generation Database Interfaces: A Survey of LLM-based Text-to-SQL](https://mlsanity.com/paper/2406.08426/) (2024)
- [OpenSearch-SQL: Enhancing Text-to-SQL with Dynamic Few-shot and Consistency Alignment](https://mlsanity.com/paper/2502.14913/) (2025)
- [Knowledge-to-SQL: Enhancing SQL Generation with Data Expert LLM](https://mlsanity.com/paper/2402.11517/) (2024)
- [Text2SQL is Not Enough: Unifying AI and Databases with TAG](https://mlsanity.com/paper/2408.14717/) (2024)
- [NL2SQL-BUGs: A Benchmark for Detecting Semantic Errors in NL2SQL Translation](https://mlsanity.com/paper/2503.11984/) (2025)
- [Supporting Our AI Overlords: Redesigning Data Systems to be Agent-First](https://mlsanity.com/paper/2509.00997/) (2025)
- [Is Long Context All You Need? Leveraging LLM's Extended Context for NL2SQL](https://mlsanity.com/paper/2501.12372/) (2025)
- [Agentar-Scale-SQL: Advancing Text-to-SQL through Orchestrated Test-Time Scaling](https://mlsanity.com/paper/2509.24403/) (2025)
- [Pneuma: Leveraging LLMs for Tabular Data Representation and Retrieval in an End-to-End System](https://mlsanity.com/paper/2504.09207/) (2025)
- [AgenticData: An Agentic Data Analytics System for Heterogeneous Data](https://mlsanity.com/paper/2508.05002/) (2025)
- [Reliable Text-to-SQL with Adaptive Abstention](https://mlsanity.com/paper/2501.10858/) (2025)
- [R-Bot: An LLM-based Query Rewrite System](https://mlsanity.com/paper/2412.01661/) (2024)
- [NameGuess: Column Name Expansion for Tabular Data](https://mlsanity.com/paper/2310.13196/) (2023)
- [Pervasive Annotation Errors Break Text-to-SQL Benchmarks and Leaderboards](https://mlsanity.com/paper/2601.08778/) (2026)
- [Knowledge Base Construction for Knowledge-Augmented Text-to-SQL](https://mlsanity.com/paper/2505.22096/) (2025)
- [Multi-Turn Interactions for Text-to-SQL with Large Language Models](https://mlsanity.com/paper/2408.11062/) (2024)

## Built on by (most cited first, 7 of 7)

- [Generating Expressive and Customizable Evals for Timeseries Data Analysis Agents with AgentFuel](https://mlsanity.com/paper/2603.12483/) (2026)
- [Structure-Grounded Knowledge Retrieval via Code Dependencies for Multi-Step Data Reasoning](https://mlsanity.com/paper/2604.10516/) (2026)
- [Sophrosyne: Agentic Exploration of Relational Data Systems Needs Moderation](https://mlsanity.com/paper/2605.30862/) (2026)
- [Agentic Data Environments](https://mlsanity.com/paper/2607.07397/) (2026)
- [From Test-Time Scaling to Reusable Memory: Measuring Crystallization in Text-to-SQL](https://mlsanity.com/paper/2608.07213/) (2026)
- [Towards Reliable AI Data Scientists: Data Agents with Workflow Harnesses](https://mlsanity.com/paper/2609.35255/) (2026)
- [CoverSynth: Coverage-Guided Generation of Retrieval Evaluation Sets](https://mlsanity.com/paper/2610.15409/) (2026)

## Abstract (arXiv)

Natural language to SQL (NL2SQL) translation enables non-expert users to query relational databases through natural language. Recently, NL2SQL agents, powered by the reasoning capabilities of Large Language Models (LLMs), have significantly advanced NL2SQL translation. Nonetheless, NL2SQL agents still make mistakes when faced with large-scale real-world databases because they lack knowledge of how to correctly leverage the underlying data (e.g., knowledge about the intent of each column) and form misconceptions about the data when querying it, leading to errors. Prior work has studied generating facts about the database to provide more context to NL2SQL agents, but such approaches simply restate database contents without addressing the agent's misconceptions. In this paper, we propose Tk-Boost, a bolt-on framework for augmenting any NL2SQL agent with tribal knowledge: knowledge that corrects the agent's misconceptions in querying the database accumulated through experience using the database. To accumulate experience, Tk-Boost first asks the NL2SQL agent to answer a few queries on the database, identifies the agent's misconceptions by analyzing its mistakes on the database, and generates tribal knowledge to address them. To enable accurate retrieval, Tk-Boost indexes this knowledge with applicability conditions that specify the query features for which the knowledge is useful. When answering new queries, Tk-Boost uses this knowledge to provide feedback to the NL2SQL agent, resolving the agent's misconceptions during SQL generation, and thus improving the agent's accuracy. Extensive experiments across the BIRD and Spider 2.0 benchmarks with various NL2SQL agents shows Tk-Boost improves NL2SQL agents accuracy by up to 16.9% on Spider 2.0 and 13.7% on BIRD