mlsanityData to 7 Oct 2026

Answer Only as Precisely as Justified: Calibrated Claim-Level Specificity Control for Agentic Systems

arXiv 2604.17487 ↗First posted 19 Apr 2026cs.CLcited by 0

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

Agentic systems often fail not by being entirely wrong, but by being too precise: a response may be generally useful while particular claims exceed what the evidence supports. We study this failure mode as overcommitment control and introduce compositional selective specificity (CSS), a post-generation layer that decomposes an answer into claims, proposes coarser backoffs, and emits each claim at the most specific calibrated level that appears admissible. The method is designed to express uncertainty as a local semantic backoff rather than as a whole-answer refusal. Across a full LongFact run and HotpotQA pilots, calibrated CSS improves the risk-utility trade-off of fixed drafts. On the full LongFact run, it raises overcommitment-aware utility from 0.846 to 0.913 relative to the no-CSS output while achieving 0.938 specificity retention. These results suggest that claim-level specificity control is a useful uncertainty interface for agentic systems and a target for future distribution-free validity layers.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 10 of its 14 references that are in the map. First posted on arXiv 19 Apr 2026.

Builds on

10 of 14 references.

References · 10

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  2. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  3. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,259
  4. 2017Selective Classification for Deep Neural Networkscited by 799
  5. 2021A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantificationcited by 777
  6. 2023Chain-of-Verification Reduces Hallucination in Large Language Modelscited by 317
  7. 2022MRKL Systems: A modular, neuro-symbolic architecture that combines large language models, external knowledge sources and discrete reasoningcited by 153
  8. 2024Language Models with Conformal Factuality Guaranteescited by 128
  9. 2025Conformal Linguistic Calibration: Trading-off between Factuality and Specificitycited by 20
  10. 2026When Should LLMs Be Less Specific? Selective Abstraction for Reliable Long-Form Text Generationcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026EAVer: Long-Form Factuality Verification as an End-to-End Agentic Policy
  2. 2026Bayesian control for coding agents
  3. 2026Budgeted Act-or-Defer Multi-Agent LLM Deliberation with Local Reliability Bounds
  4. 2026When Agents Disagree With Themselves: Behavioral Consistency as an Uncertainty Signal for LLM Agents
  5. 2026Partial Evidence Bench: Benchmarking Authorization-Limited Evidence in Agentic Systems
  6. 2026Local verification cannot detect non-transportability: a cohomological theory of context preservation in agentic reasoning
  7. 2026Think Through Uncertainty: Improving Long-Form Generation Factuality via Reasoning Calibration
  8. 2026Proper Scoring Rules for Agentic Uncertainty Quantification