# Soft Measures for Extracting Causal Collective Intelligence

arXiv: https://arxiv.org/abs/2409.18911 · HTML: https://mlsanity.com/paper/2409.18911/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 62 references that are in the map.
- First posted on arXiv 27 Sep 2024.

## Builds on (most cited first, 8 of 8)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [BLEURT: Learning Robust Metrics for Text Generation](https://mlsanity.com/paper/2004.04696/) (2020)
- [A Survey on Graph Kernels](https://mlsanity.com/paper/1903.11835/) (2019)
- [Elo Uncovered: Robustness and Best Practices in Language Model Evaluation](https://mlsanity.com/paper/2311.17295/) (2023)
- [A Labeled Array Distance Metric for Measuring Image Segmentation Quality](https://mlsanity.com/paper/2406.07851/) (2024)

## Built on by (most cited first, 1 of 1)

- [The Agentic Leash: Extracting Causal Feedback Fuzzy Cognitive Maps with LLMs](https://mlsanity.com/paper/2601.00097/) (2025)

## Abstract (arXiv)

Understanding and modeling collective intelligence is essential for addressing complex social systems. Directed graphs called fuzzy cognitive maps (FCMs) offer a powerful tool for encoding causal mental models, but extracting high-integrity FCMs from text is challenging. This study presents an approach using large language models (LLMs) to automate FCM extraction. We introduce novel graph-based similarity measures and evaluate them by correlating their outputs with human judgments through the Elo rating system. Results show positive correlations with human evaluations, but even the best-performing measure exhibits limitations in capturing FCM nuances. Fine-tuning LLMs improves performance, but existing measures still fall short. This study highlights the need for soft similarity measures tailored to FCM extraction, advancing collective intelligence modeling with NLP.