mlsanityData to 7 Oct 2026

Localizing RL-Induced Tool Use to a Single Crosscoder Feature

arXiv 2606.26474 ↗First posted 25 Jun 2026cs.LG cs.AIcited by 0

On the map: LLM agents and tool use → Reinforcement learning for LLM agents

Abstract (arXiv)

Fine-tuning through RL reshapes the internal representations of language models to enable agentic behaviors such as tool use, yet the mechanistic basis of these changes remains poorly understood. While RL substantially improves structured tool-call generation, it is unclear which features emerge, which are preserved, and whether identified features can be leveraged for retraining-free behavioral control. In this work, we show that textitDedicated Feature Crosscoders (DFC) isolate a compact set of RL-specific features that mediate tool-calling capability in textttQwen2.5-3B. Across a 48-crosscoder hyperparameter sweep, encode-decode reconstruction improves the RL model's tool correctness by +31.1 ± 9.7 pp and passively transfers tool-calling ability to the frozen base model by +6.8 ± 5.0 pp which we call a textitcapability spillover. Our findings show that DFC partitioning concentrates RL-introduced capability into a minimal, steerable feature set that enables runtime behavioral control of agentic LLMs.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reinforcement learning for LLM agents, one of 16 directions in LLM agents and tool use. Builds on 10 of its 17 references that are in the map. First posted on arXiv 25 Jun 2026.

Builds on

10 of 17 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025ToolRL: Reward is All Tool Learning Needs16 Apr

Other references · 9

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2024Qwen2.5 Technical Reportcited by 4,713
  3. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,359
  4. 2024The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scalecited by 1,104
  5. 2022Toy Models of Superpositioncited by 928
  6. 2026Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnetcited by 175
  7. 2024Knowledge Boundary of Large Language Models: A Surveycited by 36
  8. 2026Cross-Architecture Model Diffing with Crosscoders: Unsupervised Discovery of Differences Between LLMscited by 12
  9. 2026Delta-Crosscoder: Robust Crosscoder Model Diffing in Narrow Fine-Tuning Regimescited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Spurious Tool Use: When RL Agents Learn the Wrong Reason to Act
  2. 2026Controlling Tool Use with Heading-Specific Activation Steering
  3. 2026On Effectiveness and Efficiency of Agentic Tool-calling and RL Training
  4. 2026Toolcompass: Guiding Tool Trialing, Not Suppressing It
  5. 2025ToolACE-R: Model-aware Iterative Training and Adaptive Refinement for Tool Learning
  6. 2026NTILC: Neural Tool Invocation via Learned Compression
  7. 2026Why Multi-Step Tool-Use Reinforcement Learning Collapses and How Supervisory Signals Fix It
  8. 2025Tool Zero: Training Tool-Augmented LLMs via Pure RL from Scratch