mlsanityData to 14 Oct 2026

Policy as Code: A Coroutine-Bridge Harness for Fast-Reasoning Reliability on CAR-bench

arXiv 2609.29251 ↗First posted 24 Sep 2026cs.AI cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

CAR-bench evaluates whether tool-using agents stay reliable under real-world uncertainty, executing every tool inside the evaluator so that each tool-result exchange is a separate agent round-trip. A conventional next-action agent can batch parallel tool calls, but a chain of dependent calls costs it one model call per round of results. We present a coroutine-bridge harness in which the model's only action is to emit a Python program that blocks and resumes in place across evaluator tool exchanges. This decouples model invocation from tool round-trips: on the public test split the agent uses a median of two model calls against seven agent turns per task, resolving a full multi-turn task in a median of 1.8 s of model latency on Cerebras gpt-oss-120b. Because the action surface is executable code, deterministic CAR-bench policies are encoded directly as logic in the tool layer rather than as prompt rules, enforcing compliance at zero reasoning cost. On the official hidden evaluation the harness won Track 2 with 60.0% Pass^3, 4.5x the organizer baseline, at the lowest estimated cost and the fastest median task latency (3.14 s) of any entry scoring above that baseline; the same unchanged harness reproduced an identical 60.0% Pass^3 on GPT-5.5 in the Open track, matching frontier-model agents. A single static prompt, appended with per-task state at the tail, stays byte-identical across calls and across tasks: the frozen submission prompt served 78% of input tokens from cache (86.6% across its warm tail), against 73% over a three-week development corpus in which prompt edits repeatedly reset the cache. This compounds the few-call design into a small fraction of nominal input compute.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 3 of its 8 references that are in the map. First posted on arXiv 24 Sep 2026.

Builds on

3 of 8 references.

References · 3

  1. 2024Executable Code Actions Elicit Better LLM Agentscited by 564
  2. 2026CAR-bench: Evaluating the Consistency and Limit-Awareness of LLM Agents under Real-World Uncertaintycited by 8
  3. 2026Token-Oriented Object Notation vs JSON: A Benchmark of Plain and Constrained Decoding Generationcited by 3

Built on by

Not cited yet.