# Counterfactual-based Agent Influence Ranker for Agentic AI Workflows

arXiv: https://arxiv.org/abs/2510.25612 · HTML: https://mlsanity.com/paper/2510.25612/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use.
- Builds on 9 of its 68 references that are in the map.
- First posted on arXiv 29 Oct 2025.

## Placed in

- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)

## Builds on (most cited first, 9 of 9)

- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Certifying LLM Safety against Adversarial Prompting](https://mlsanity.com/paper/2309.02705/) (2023)
- [Fully Autonomous AI Agents Should Not be Developed](https://mlsanity.com/paper/2502.02649/) (2025)
- [Understanding Individual Agent Importance in Multi-Agent System via Counterfactual Reasoning](https://mlsanity.com/paper/2412.15619/) (2024)
- [Developer Experiences with a Contextualized AI Coding Assistant: Usability, Expectations, and Outcomes](https://mlsanity.com/paper/2311.18452/) (2023)
- [Centrality Measures in Complex Networks: A Survey](https://mlsanity.com/paper/2011.07190/) (2020)
- [Deep Clustering for Data Cleaning and Integration](https://mlsanity.com/paper/2305.13494/) (2023)

## Built on by (most cited first, 2 of 2)

- [RIFT-Bench: Dynamic Red-teaming For Agentic AI Systems](https://mlsanity.com/paper/2606.23927/) (2026)
- [Search-G1: Grounded Search Agents via Representation-Based Intrinsic Rewards](https://mlsanity.com/paper/2608.07531/) (2026)

## Abstract (arXiv)

An Agentic AI Workflow (AAW), also known as an LLM-based multi-agent system, is an autonomous system that assembles several LLM-based agents to work collaboratively towards a shared goal. The high autonomy, widespread adoption, and growing interest in such AAWs highlight the need for a deeper understanding of their operations, from both quality and security aspects. To this day, there are no existing methods to assess the influence of each agent on the AAW's final output. Adopting techniques from related fields is not feasible since existing methods perform only static structural analysis, which is unsuitable for inference time execution. We present Counterfactual-based Agent Influence Ranker (CAIR) - the first method for assessing the influence level of each agent on the AAW's output and determining which agents are the most influential. By performing counterfactual analysis, CAIR provides a task-agnostic analysis that can be used both offline and at inference time. We evaluate CAIR using an AAWs dataset of our creation, containing 30 different use cases with 230 different functionalities. Our evaluation showed that CAIR produces consistent rankings, outperforms baseline methods, and can easily enhance the effectiveness and relevancy of downstream tasks.