mlsanityData to 7 Oct 2026

When Agents Look the Same: Quantifying Distillation-Induced Similarity in Tool-Use Behaviors

arXiv 2604.21255 ↗First posted 23 Apr 2026cs.CLcited by 1

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

Model distillation is a primary driver behind the rapid progress of LLM agents, yet it often leads to behavioral homogenization. Many emerging agents share nearly identical reasoning steps and failure modes, suggesting they may be distilled echoes of a few dominant teachers. Existing metrics, however, fail to distinguish mandatory behaviors required for task success from non-mandatory patterns that reflect a model's autonomous preferences. We propose two complementary metrics to isolate non-mandatory behavioral patterns: Response Pattern Similarity (RPS) for verbal alignment and Action Graph Similarity (AGS) for tool-use habits modeled as directed graphs. Evaluating 18 models from 8 providers on τ-Bench and τ2-Bench against Claude Sonnet 4.5 (thinking), we find that within-family model pairs score 5.9 pp higher in AGS than cross-family pairs, and that Kimi-K2 (thinking) reaches 82.6% Snode and 94.7% Sdep, exceeding Anthropic's own Opus 4.1. A controlled distillation experiment further confirms that AGS distinguishes teacher-specific convergence from general improvement. RPS and AGS capture distinct behavioral dimensions (Pearson r = 0.491), providing complementary diagnostic signals for behavioral convergence in the agent ecosystem. Our code is available at https://github.com/Syuchin/AgentEcho.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 27 of its 72 references that are in the map. First posted on arXiv 23 Apr 2026.

Builds on

27 of 72 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025Exploring Autonomous Agents: A Closer Look at Why They Fail When Completing Tasks18 Aug
  2. 2025Distilling LLM Agent into Small Models with Retrieval and Code Tools23 May
  3. 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains17 Jun

Other references · 24

  1. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  2. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  3. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  4. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
  5. 2020MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformerscited by 1,414
  6. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,302
  7. 2019TinyBERT: Distilling BERT for Natural Language Understandingcited by 1,153
  8. 2023Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizescited by 710
  9. 2024WildChat: 1M ChatGPT Interaction Logs in the Wildcited by 598
  10. 2023API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMscited by 574
  11. 2023NLP Evaluation in trouble: On the Need to Measure LLM Data Contamination for each Benchmarkcited by 339
  12. 2022Data Contamination: From Memorization to Exploitationcited by 234
12 more
  1. 2024Trial and Error: Exploration-Based Trajectory Optimization for LLM Agentscited by 194
  2. 2024A Note on Shumailov et al. (2024): `AI Models Collapse When Trained on Recursively Generated Data'cited by 114
  3. 2023The Curious Decline of Linguistic Diversity: Training Language Models on Synthetic Textcited by 97
  4. 2024Generalization v.s. Memorization: Tracing Language Models' Capabilities Back to Pretraining Datacited by 83
  5. 2025TRAJECT-Bench:A Trajectory-Aware Benchmark for Evaluating Agentic Tool Usecited by 46
  6. 2025Unveiling the Key Factors for Distilling Chain-of-Thought Reasoningcited by 40
  7. 2024Evading Data Contamination Detection for Language Models is (too) Easycited by 30
  8. 2024PromptKD: Distilling Student-Friendly Knowledge for Generative Language Models via Prompt Tuningcited by 24
  9. 2025Butterfly Effects in Toolchains: A Comprehensive Analysis of Failed Parameter Filling in LLM Tool-Agent Systemscited by 17
  10. 2025How Contaminated Is Your Benchmark? Quantifying Dataset Leakage in Large Language Models with Kernel Divergencecited by 13
  11. 2025Student-Centered Distillation Narrows the Agentic Gap Between Small and Large LLMscited by 11
  12. 2025Quantification of Large Language Model Distillationcited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2026AgentHabit: Characterizing Distinct Behaviors of Agents on Everyday Tasks
  2. 2026Measuring Cross-Task Behavioral Consistency in Language Model Agents
  3. 2026Set-shifting Behavioral Test for Harnessed Agents
  4. 2026Model-Adaptive Tool Necessity Reveals the Knowing-Doing Gap in LLM Tool Use
  5. 2026On the Behavioral Traits of LLM Agents
  6. 2026ToolAnchor: Anchoring Counterfactual Context to Boost Agentic Tool-use Capability
  7. 2026Distilling Agentic Systems: A Roadmap across Models, Artifacts, and Harnesses
  8. 2026Agent trajectories as programs: fingerprinting and programming coding-agent behavior