# Measuring Similarity in Causal Graphs: A Framework for Semantic and Structural Analysis

arXiv: https://arxiv.org/abs/2503.11046 · HTML: https://mlsanity.com/paper/2503.11046/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 55 references that are in the map.
- First posted on arXiv 14 Mar 2025.

## Builds on (most cited first, 15 of 15)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [On LLMs-Driven Synthetic Data Generation, Curation, and Evaluation: A Survey](https://mlsanity.com/paper/2406.15126/) (2024)
- [Increasing Diversity While Maintaining Accuracy: Text Data Generation with Large Language Models and Human Interventions](https://mlsanity.com/paper/2306.04140/) (2023)
- [Propagation Kernels](https://mlsanity.com/paper/1410.3314/) (2014)
- [Evolution of Semantic Similarity -- A Survey](https://mlsanity.com/paper/2004.13820/) (2020)
- [Graph Kernels: A Survey](https://mlsanity.com/paper/1904.12218/) (2019)
- [Metrics for Graph Comparison: A Practitioner's Guide](https://mlsanity.com/paper/1904.07414/) (2019)
- [Directed Diversity: Leveraging Language Embedding Distances for Collective Creativity in Crowd Ideation](https://mlsanity.com/paper/2101.06030/) (2021)
- [Separation-based distance measures for causal graphs](https://mlsanity.com/paper/2402.04952/) (2024)
- [From Text to Map: A System Dynamics Bot for Constructing Causal Loop Diagrams](https://mlsanity.com/paper/2402.11400/) (2024)
- [The Resistance Perturbation Distance: A Metric for the Analysis of Dynamic Networks](https://mlsanity.com/paper/1605.01091/) (2016)

## Abstract (arXiv)

Causal graphs are commonly used to understand and model complex systems. Researchers often construct these graphs from different perspectives, leading to significant variations for the same problem. Comparing causal graphs is, therefore, essential for evaluating assumptions, integrating insights, and resolving disagreements. The rise of AI tools has further amplified this need, as they are increasingly used to generate hypothesized causal graphs by synthesizing information from various sources such as prior research and community inputs, providing the potential for automating and scaling causal modeling for complex systems. Similar to humans, these tools also produce inconsistent results across platforms, versions, and iterations. Despite its importance, research on causal graph comparison remains scarce. Existing methods often focus solely on structural similarities, assuming identical variable names, and fail to capture nuanced semantic relationships, which is essential for causal graph comparison. We address these gaps by investigating methods for comparing causal graphs from both semantic and structural perspectives. First, we reviewed over 40 existing metrics and, based on predefined criteria, selected nine for evaluation from two threads of machine learning: four semantic similarity metrics and five learning graph kernels. We discuss the usability of these metrics in simple examples to illustrate their strengths and limitations. We then generated a synthetic dataset of 2,000 causal graphs using generative AI based on a reference diagram. Our findings reveal that each metric captures a different aspect of similarity, highlighting the need to use multiple metrics.