mlsanityData to 14 Oct 2026

When Should Enterprise Policy Decisions Be Learned, and When Should They Be Reasoned?

arXiv 2610.17643 ↗First posted 13 Oct 2026cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

An enterprise agent that answers under a written policy must choose an action before it says anything: answer, ask for a missing fact, ask for confirmation, call one or several tools, escalate, or refuse. Holding one 27B Korean enterprise model fixed, we vary whether the decision is a generated JSON object or a single constrained token, and whether the model may reason first. On 2800 in-distribution items we did not detect an accuracy difference between the two deployed paths (93.46% vs 94.14%), while they differ about tenfold in restart stability (3.00% of actions flip vs 0.29%); they also differ in reasoning, so this is an operational comparison. On a sealed benchmark of 300 items whose labels come from a hidden executable policy, reasoning is worth +17.3 points [+12.3, +23.0] on exact action. With reasoning off, the format difference was 0.0 points [-4.3, +4.0], a failure to detect an advantage for either interface, not equivalence; with reasoning on it is unresolved. Against a token-matched random control, boundary-targeted supervision moves the two covered boundaries by +17.5 and +20.0 points and leaves a held-out boundary unchanged. In a 3-seed training x reasoning factorial, the sealed-rendering interaction is -4.9 [-8.1, -1.7], consistent with partial substitution. All supervision was Korean. On independently rendered English versions, the model is 0.82 points lower [-2.68, +0.93] with reasoning on, the same order as re-rendering in Korean; the reasoning-off gain from targeted supervision is not stable in either language, so we do not claim transfer. The original text stays private as the confirmatory set. An open-model re-rendering made after these results reproduced the reasoning effect and the English verdict; the interaction kept its sign but did not replicate under item-clustered intervals. We release the model and that re-rendering with gold labels.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 33 of its 40 references that are in the map. First posted on arXiv 13 Oct 2026.

Builds on

33 of 40 references.

References · 33

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  3. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  4. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  5. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  6. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
  7. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 2,006
  8. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,986
  9. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,947
  10. 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,402
  11. 2025s1: Simple test-time scalingcited by 1,341
  12. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,332
21 more
  1. 2022STaR: Bootstrapping Reasoning With Reasoningcited by 940
  2. 2023API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMscited by 591
  3. 2021PICARD: Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Modelscited by 411
  4. 2024WorkArena: How Capable Are Web Agents at Solving Common Knowledge Work Tasks?cited by 363
  5. 2020A Simple Language Model for Task-Oriented Dialoguecited by 332
  6. 2024Aya Model: An Instruction Finetuned Open-Access Multilingual Language Modelcited by 295
  7. 2024ToolSandbox: A Stateful, Conversational, Interactive Evaluation Benchmark for LLM Tool Use Capabilitiescited by 295
  8. 2023Efficient Guided Generation for Large Language Modelscited by 200
  9. 2023Grammar-Constrained Decoding for Structured NLP Tasks without Finetuningcited by 197
  10. 2025AdaptThink: Reasoning Models Can Learn When to Thinkcited by 147
  11. 2024XGrammar: Flexible and Efficient Structured Generation Engine for Large Language Modelscited by 75
  12. 2024Multilingual Instruction Tuning With Just a Pinch of Multilingualitycited by 73
  13. 2025JSONSchemaBench: A Rigorous Benchmark of Structured Outputs for Language Modelscited by 71
  14. 2025Learning When to Think: Shaping Adaptive Reasoning in R1-Style Models via Multi-Stage RLcited by 62
  15. 2026Reasoning Theater: Disentangling Model Beliefs from Chain-of-Thoughtcited by 40
  16. 2024Zero-shot cross-lingual transfer in instruction tuning of large language modelscited by 11
  17. 2026Same Signal, Opposite Meaning: Direction-Informed Adaptive Learning for LLM Agentscited by 8
  18. 2026OrgForge-IT: A Verifiable Synthetic Benchmark for LLM-Based Insider Threat Detectioncited by 5
  19. 2026Efficient Test-Time Scaling via Temporal Reasoning Aggregationcited by 3
  20. 2026Ground Truth First: A Longitudinal Evaluation Instrument for Agent Memory, and the Tenure Crossover in Memory-Architecture Rankingscited by 3
  21. 2026Learning When to Think: Adaptive Reasoning for Test-Time Compute Allocationcited by 2

Built on by

Not cited yet.