mlsanityData to 14 Oct 2026

Benchmarking graph construction by large language models for coherence-driven inference

arXiv 2502.13953 ↗First posted 19 Feb 2025cs.AIcited by 2

Not on the map. Why?

Abstract (arXiv)

We devise an algorithm to generate propositions that objectively instantiate graphs supporting coherence-driven inference. We also benchmark the ability of large language models (LLMs) to reconstruct coherence graphs from (a simple transformation of) propositions expressed in natural language, with promising results from a single prompt to reasoning-optimized LLMs. For example, o1/3/4-mini achieve perfect reconstruction half of the time on sparse graphs. Coherence-driven inference on consistency evaluations by LLMs may advance machine cognition capabilities.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 28 of its 105 references that are in the map. First posted on arXiv 19 Feb 2025.

Builds on

28 of 105 references.

References · 28

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  4. 2024GPT-4o System Cardcited by 4,588
  5. 2023Progress measures for grokking via mechanistic interpretabilitycited by 839
  6. 2024Phi-4 Technical Reportcited by 711
  7. 2023Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoningcited by 391
  8. 2022Language Models Are Greedy Reasoners: A Systematic Formal Analysis of Chain-of-Thoughtcited by 337
  9. 2022FOLIO: Natural Language Reasoning with First-Order Logiccited by 235
  10. 2023LINC: A Neurosymbolic Approach for Logical Reasoning by Combining Language Models with First-Order Logic Proverscited by 168
  11. 2024Infini-gram: Scaling Unbounded n-gram Language Models to a Trillion Tokenscited by 153
  12. 2023SatLM: Satisfiability-Aided Language Models Using Declarative Promptingcited by 102
16 more
  1. 2024LLMs Still Can't Plan; Can LRMs? A Preliminary Evaluation of OpenAI's o1 on PlanBenchcited by 94
  2. 2019On the Existence of Simpler Machine Learning Modelscited by 89
  3. 2021From Statistical Relational to Neurosymbolic Artificial Intelligence: a Surveycited by 72
  4. 2024On Limitations of the Transformer Architecturecited by 63
  5. 2021AR-LSAT: Investigating Analytical Reasoning of Textcited by 51
  6. 2024Transformers on Markov Data: Constant Depth Sufficescited by 27
  7. 2024A Closer Look at Classification Evaluation Metrics and a Critical Reflection of Common Evaluation Practicecited by 26
  8. 2024On the Representational Capacity of Neural Language Models with Chain-of-Thought Reasoningcited by 26
  9. 2014Learning Latent Block Structure in Weighted Networkscited by 23
  10. 2024Machine learning and information theory concepts towards an AI Mathematiciancited by 20
  11. 2023Getting from Generative AI to Trustworthy AI: What LLMs might learn from Cyccited by 19
  12. 2018Spectral Sparsification of Hypergraphscited by 10
  13. 2024Prospects for inconsistency detection using large language models and sheavescited by 10
  14. 2024NLP-based Regulatory Compliance -- Using GPT 4.0 to Decode Regulatory Documentscited by 4
  15. 2023Goodness of fit testing based on graph functionals for homogenous Erd"os Renyi graphscited by 1
  16. 2024A Neurosymbolic Fast and Slow Architecture for Graph Coloringcited by 1

Built on by

Cited by year

Citing papers per year: 2025: 2.

2 papers.

  1. 2025Coherence-driven inference for cybersecuritycited by 0
  2. 2025Automatic coherence-driven inference on argumentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Graph Property Inference in Small Language Models: Effects of Representation and Reasoning Strategy
  2. 2026Can We Trust LLM's Logic? Quantifying Uncertainty, Coherence, and Robustness via a Graph-Based Framework
  3. 2025Can an LLM Induce a Graph? Investigating Memory Drift and Context Length
  4. 2026Coherence Under Commitment: Probing Generalization and Vacuous Memorization in LLM Logical Reasoning
  5. 2024GraphInstruct: Empowering Large Language Models with Graph Understanding and Reasoning Capability
  6. 2023Evaluating Large Language Models on Graphs: Performance Insights and Comparative Analysis
  7. 2025GraphGhost: Tracing Structures Behind Large Language Models
  8. 2025A Graph Talks, But Who's Listening? Rethinking Evaluations for Graph-Language Models