mlsanityData to 7 Oct 2026

MICE for CATs: Model-Internal Confidence Estimation for Calibrating Agents with Tools

arXiv 2504.20168 ↗First posted 28 Apr 2025cs.CL cs.AI cs.LGcited by 15

On the map: LLM agents and tool use → Tool learning and function calling

Abstract (arXiv)

Tool-using agents that act in the world need to be both useful and safe. Well-calibrated model confidences can be used to weigh the risk versus reward of potential actions, but prior work shows that many models are poorly calibrated. Inspired by interpretability literature exploring the internals of models, we propose a novel class of model-internal confidence estimators (MICE) to better assess confidence when calling tools. MICE first decodes from each intermediate layer of the language model using logitLens and then computes similarity scores between each layer's generation and the final output. These features are fed into a learned probabilistic classifier to assess confidence in the decoded output. On the simulated trial and error (STE) tool-calling dataset using Llama3 models, we find that MICE beats or matches the baselines on smoothed expected calibration error. Using MICE confidences to determine whether to call a tool significantly improves over strong baselines on a new metric, expected tool-calling utility. Further experiments show that MICE is sample-efficient, can generalize zero-shot to unseen APIs, and results in higher tool-calling utility in scenarios with varying risk levels. Our code is open source, available at https://github.com/microsoft/mice_for_cats.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Placed in Tool learning and function calling, one of 16 directions in LLM agents and tool use. Builds on 31 of its 60 references that are in the map. First posted on arXiv 28 Apr 2025.

Builds on

31 of 60 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Toolformer: Language Models Can Teach Themselves to Use Tools9 Feb

Other references · 30

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  3. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  4. 2017On Calibration of Modern Neural Networkscited by 4,860
  5. 2017Deep reinforcement learning from human preferencescited by 3,846
  6. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,157
  7. 2022Language Models (Mostly) Know What They Knowcited by 1,783
  8. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,371
  9. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
  10. 2019BERT Rediscovers the Classical NLP Pipelinecited by 1,046
  11. 2017BranchyNet: Fast Inference via Early Exiting from Deep Neural Networkscited by 591
  12. 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 579
18 more
  1. 2022Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Spacecited by 522
  2. 2020How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answeringcited by 439
  3. 2020Calibration of Pre-trained Transformerscited by 307
  4. 2024LayerSkip: Enabling Early Exit Inference and Self-Speculative Decodingcited by 302
  5. 2023Do Large Language Models Know What They Don't Know?cited by 238
  6. 2020Reducing conversational agents' overconfidence through linguistic calibrationcited by 209
  7. 2023Jump to Conclusions: Short-Cutting Transformers With Linear Transformationscited by 89
  8. 2019Calibration of Encoder Decoder Models for Neural Machine Translationcited by 67
  9. 2020Quantifying Differences in Reward Functionscited by 62
  10. 2020On the Inference Calibration of Neural Machine Translationcited by 61
  11. 2023Smooth ECE: Principled Reliability Diagrams via Kernel Smoothingcited by 59
  12. 2022Calibrated Interpretation: Confidence Estimation in Semantic Parsingcited by 33
  13. 2019Can Unconditional Language Models Recover Arbitrary Sentences?cited by 15
  14. 2014Binary Classifier Calibration: Non-parametric approachcited by 13
  15. 2022Non-Programmers Can Label Programs Indirectly via Active Examples: A Case Study with Text-to-SQLcited by 12
  16. 2022BenchCLAMP: A Benchmark for Evaluating Language Models on Syntactic and Semantic Parsingcited by 12
  17. 2020Discovering Useful Sentence Representations from Large Pretrained Language Modelscited by 6
  18. 2023Did You Mean...? Confidence-based Trade-offs in Semantic Parsingcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 4; 2026: 11.

15 papers.

Sort:

By area:

  1. 2025LLM Microscope: What Model Internals Reveal About Answer Correctness and Context Utilizationcited by 11
  2. 2026REFLEX with Jev for Efficient Selective Control in LLM Agentscited by 9
  3. 2025Model Internal Sleuthing: Finding Lexical Identity and Inflectional Features in Modern Language Modelscited by 5
  4. 2025SimBA: Simplifying Benchmark Analysis Using Performance Matrices Alonecited by 5
  5. 2025Localized Calibrated Uncertainty in Code Language Modelscited by 3
  6. 2026Uncertainty Decomposition for Clarification Seeking in LLM Agentscited by 3
  7. 2026Do LLMs Know Tool Irrelevance? Demystifying Structural Alignment Bias in Tool Invocationscited by 2
  8. 2026Critic Experience Bank: Self-Evolving Step-Level Confidence Estimation for LLM Agentscited by 2
  9. 2026The ACUTE Protocol: Operationalizing Language Model Activations for Better Calibration, Utility, and Trustcited by 1
  10. 2026How to Speculate about Uncertainty in Agentic Coding? A Draft-Model Gate Methodcited by 1
  11. 2026Mobile-Aptus: Confidence-Driven Proactive and Robust Interaction in MLLM-based Mobile-Using Agentscited by 0
  12. 2026Harnessing the Latent Space: From Steering Vectors to Model Calibrators for Control and Trustcited by 0
3 more
  1. 2026Do Agents Know When They Succeed? Calibrating Agent Confidence from Internal Representationscited by 0
  2. 2026When Tools Silently Lie: Evaluating and Mitigating Blind Compliance in Tool-Augmented Data Agentscited by 0
  3. 2026What Can Component-Replacement Evidence Establish? A Critical Scoping Review of Local Decisions in LLM Agentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026The Confidence Dichotomy: Analyzing and Mitigating Miscalibration in Tool-Use Agents
  2. 2025Knowing What You Know Is Not Enough: Large Language Model Confidences Don't Align With Their Actions
  3. 2026ToolMenuBench: Benchmarking Tool-Menu Filtering Strategies for Reliable and Efficient LLM Agents
  4. 2025Confidence Calibration in Vision-Language-Action Models
  5. 2026ToolFailBench: Diagnosing Tool-Use Failures in LLM Agents
  6. 2024Confidence Under the Hood: An Investigation into the Confidence-Probability Alignment in Large Language Models
  7. 2026When Simulation Lies: A Sim-to-Real Benchmark and Domain-Randomized RL Recipe for Tool-Use Agents
  8. 2026Proxy Confidence: Auditing Black-Box LLM Agents with a Surrogate's Log-Probabilities