Beyond Static Snapshots: A Grounded Evaluation Framework for Language Models at the Agentic Frontier
On the map: LLM reasoning → Reinforcement learning for reasoning
also in 2 more
Abstract (arXiv)
We argue that current evaluation frameworks for large language models (LLMs) suffer from four systematic failures that make them structurally inadequate for deployed, agentic systems: distributional, temporal, scope, and process invalidity. These failures compound in RLHF, making reward hacking a predictable consequence of evaluation design rather than an unpredictable training pathology, and RLHF's dual-model architecture imposes a hardware barrier limiting evaluation reproducibility. We propose the Grounded Continuous Evaluation (GCE) framework and present ISOPro as a reference implementation. ISOPro replaces the learned reward model with a deterministic verifier, eliminating reward hacking by construction in verifiable-reward domains, and updates LoRA adapters on CPU, reducing the hardware barrier by an order of magnitude. We validate ISOPro across three architectures (Qwen 2.5 3B, Llama 3.2 3B, Gemma 2 2B) and two domains (scheduling, MBPP), with a head-to-head matched-compute comparison against GRPO-LoRA. Across twelve cells, ISOPro produces the largest absolute capability gains (+25.6, +22.2, +16.0pp) at mean delta +9.0pp and worst-case regression -5.6pp; GRPO-LoRA at consumer-budget hyperparameters reaches a smaller peak gain (+8.5pp), deeper worst-case regression (-10pp), and mean delta -1.5pp. Held-out compositional generalization on MBPP reaches 40% for ISOPro on two of three architectures (including a 0% to 40% bootstrap on Qwen 2.5 3B), against 20% for GRPO-LoRA on one of three. We characterize a buffer-skew failure mode in which the implicit curriculum can erode pre-existing tier capability under three preconditions, with three corresponding mitigations. The work is situated alongside DeepSeek-R1's GRPO, which arrived at the same architectural conclusion at scale: for verifiable-reward domains, the verifier is the reward signal.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 13 of its 17 references that are in the map. First posted on arXiv 19 Apr 2026.
Builds on
13 of 17 references.
Closest ancestors · 3
Same direction, newest first
- 2023Let's Verify Step by Step31 May
- 2022Constitutional AI: Harmlessness from AI Feedback15 Dec
- 2017Proximal Policy Optimization Algorithms20 Jul
Other references · 10
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2022Training language models to follow instructions with human feedbackcited by 15,971
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
- 2024Qwen2.5 Technical Reportcited by 4,942
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
- 2020Learning to summarize from human feedbackcited by 2,588
- 2022Emergent Abilities of Large Language Modelscited by 2,512
- 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,998
- 2022Scaling Laws for Reward Model Overoptimizationcited by 1,095
- 2024Sycophancy to Subterfuge: Investigating Reward-Tampering in Large Language Modelscited by 178
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026UniACE: A Unified Framework for Evaluating LLM Agentic Capabilities
- 2026AgentGym2: Benchmarking Large Language Model Agents in De-Idealized Real-World Environments
- 2023AgentSims: An Open-Source Sandbox for Large Language Model Evaluation
- 2026AgentCompass: A Unified Evaluation Infrastructure for Agent Capabilities
- 2026BabelArena: A Large-Scale Multilingual Benchmark for LLM Agents
- 2026EvalBuilder: Can LLMs Build Efficient Benchmarks?
- 2026Beyond the Leaderboard: A Synthesis of Tool-Use, Planning, and Reasoning Failures in Large Language Model Agents
- 2026Reward Hacking Benchmark: Measuring Exploits in LLM Agents with Tool Use