mlsanityData to 7 Oct 2026

Graph-Grounded LLMs: Leveraging Graphical Function Calling to Minimize LLM Hallucinations

arXiv 2503.10941 ↗First posted 13 Mar 2025cs.AI cs.LG cs.ROcited by 4

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

The adoption of Large Language Models (LLMs) is rapidly expanding across various tasks that involve inherent graphical structures. Graphs are integral to a wide range of applications, including motion planning for autonomous vehicles, social networks, scene understanding, and knowledge graphs. Many problems, even those not initially perceived as graph-based, can be effectively addressed through graph theory. However, when applied to these tasks, LLMs often encounter challenges, such as hallucinations and mathematical inaccuracies. To overcome these limitations, we propose Graph-Grounded LLMs, a system that improves LLM performance on graph-related tasks by integrating a graph library through function calls. By grounding LLMs in this manner, we demonstrate significant reductions in hallucinations and improved mathematical accuracy in solving graph-based problems, as evidenced by the performance on the NLGraph benchmark. Finally, we showcase a disaster rescue application where the Graph-Grounded LLM acts as a decision-support system.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 30 of its 37 references that are in the map. First posted on arXiv 13 Mar 2025.

Builds on

30 of 37 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Tulip Agent -- Enabling LLM-Based Agents to Solve Tasks Using Large Tool Libraries31 Jul
  2. 2023SMART-LLM: Smart Multi-Agent Robot Task Planning using Large Language Models18 Sep

Other references · 28

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  2. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  3. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
  4. 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,029
  5. 2023GPT-Driver: Learning to Drive with GPTcited by 344
  6. 2022Selection-Inference: Exploiting Large Language Models for Interpretable Logical Reasoningcited by 331
  7. 2023Exploring the Potential of Large Language Models (LLMs) in Learning on Graphscited by 271
  8. 2023Can Language Models Solve Graph Problems in Natural Language?cited by 249
  9. 2023Large Language Models on Graphs: A Comprehensive Surveycited by 190
  10. 2024AnyTool: Self-Reflective, Hierarchical Agents for Large-Scale API Callscited by 121
  11. 2019WIQA: A dataset for "What if..." reasoning over procedural textcited by 71
  12. 2024A Survey of Large Language Models for Graphscited by 71
16 more
  1. 2023Towards Better Chain-of-Thought Prompting Strategies: A Surveycited by 62
  2. 2021ExplaGraphs: An Explanation Graph Generation Task for Structured Commonsense Reasoningcited by 58
  3. 2024Can we Soft Prompt LLMs for Graph Learning Tasks?cited by 52
  4. 2021Learning Knowledge Graph-based World Models of Textual Environmentscited by 31
  5. 2024LaMI: Large Language Models for Multi-Modal Human-Robot Interactioncited by 21
  6. 2024Scalable and Accurate Graph Reasoning with LLM-based Multi-Agentscited by 21
  7. 2025GFlowVLM: Enhancing Multi-step Reasoning in Vision-Language Models with Generative Flow Networkscited by 15
  8. 2021Could you give me a hint? Generating inference graphs for defeasible reasoningcited by 10
  9. 2024To Help or Not to Help: LLM-based Attentive Support for Human-Robot Group Interactionscited by 10
  10. 2023Toward Grounded Commonsense Reasoningcited by 9
  11. 2025Generalized Mission Planning for Heterogeneous Multi-Robot Teams via LLM-constructed Hierarchical Treescited by 9
  12. 2020Towards Safer Self-Driving Through Great PAIN (Physically Adversarial Intelligent Networks)cited by 6
  13. 2023Interaction-Aware Trajectory Planning for Autonomous Vehicles with Analytic Integration of Neural Networks into Model Predictive Controlcited by 5
  14. 2023KGLens: Towards Efficient and Effective Knowledge Probing of Large Language Models with Knowledge Graphscited by 5
  15. 2024Towards Scalable & Efficient Interaction-Aware Planning in Autonomous Vehicles using Knowledge Distillationcited by 4
  16. 2022Structural Properties of Optimal Fidelity Selection Policies for Human-in-the-loop Queuescited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026KGLAMP: Knowledge Graph-guided Language model for Adaptive Multi-robot Planning and Replanningcited by 2
  2. 2026Scale-Plan: Scalable Language-Enabled Task Planning for Heterogeneous Multi-Robot Teamscited by 1
  3. 2026Are Large Language Models Suitable for Graph Computation? Progress and Prospectscited by 0
  4. 2026An Exploration of Agentic Information Fusion for Test Maintenance Predictioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Exploring Graph Learning Tasks with Pure LLMs: A Comprehensive Benchmark and Investigation
  2. 2024Can Graph Descriptive Order Affect Solving Graph Problems with LLMs?
  3. 2024Graph Reasoning with Large Language Models via Pseudo-code Prompting
  4. 2024Can LLMs perform structured graph reasoning?
  5. 2026LLMs+Graphs: Toward Graph-Native, Synergistic AI Systems
  6. 2024GraphArena: Evaluating and Exploring Large Language Models on Graph Computation
  7. 2026Integrating Graphs, Large Language Models, and Agents: Reasoning and Retrieval
  8. 2025Rethinking and Benchmarking Large Language Models for Graph Reasoning