From Migration to Calibration: Preserving Agent Capabilities across Models, Jurisdictions, and Scale
Not on the map. Why?
Abstract (arXiv)
Deploying, migrating, or scaling an agent can change its model, harness, infrastructure, application, and intended users. We formulate agent calibration as standards-first adaptation: define basic-capability, technical-environment, and user-context standards; diagnose gaps; generate and apply revisions; and recheck the same standards within fixed budgets. These standard families interact across information, harness, and user-acceptance layers. Source behavior is diagnostic, not a perfect reference or capability ceiling: model replacement can turn correct answers into errors or errors into correct answers. Qualification requires all mandatory known tests, actual end-to-end deployment paths, hard predicates, and declared task/user minimums to pass; aggregate gains cannot erase hard failures. Revisions may change tools or harnesses, add demonstrations and task descriptions, or use validated target-native trajectories to train a policy, controller, or compact skill model served through the harness. Semantic checkpoints validate executed artifacts, localize repair, and revalidate dependencies. The loop exports reusable configuration or training artifacts with a qualification record, while final task outputs undergo their own checks. Independent factual evidence precedes relative preference judgment; DPO and GRPO optimize policies rather than establish truth. Frozen held-out evaluation tests generalization and compares equal-budget target-native optimization. We specify an automatic calibration tool using limited authorized user trajectories and tests as future work. The framework and tool remain proposals; confirmatory empirical validation is pending.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 46 of its 50 references that are in the map. First posted on arXiv 28 Sep 2026.
Builds on
46 of 50 references.
References · 46
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
- 2023Let's Verify Step by Stepcited by 3,598
- 2019Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learningcited by 750
- 2025GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learningcited by 449
- 2022RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learningcited by 361
- 2025ToolRL: Reward is All Tool Learning Needscited by 342
- 2025Rubrics as Rewards: Reinforcement Learning Beyond Verifiable Domainscited by 328
- 2024ProcessBench: Identifying Process Errors in Mathematical Reasoningcited by 277
- 2025AgentSpec: Customizable Runtime Enforcement for Safe and Reliable LLM Agentscited by 196
34 more
- 2026SkillOpt: Executive Strategy for Self-Evolving Agent Skillscited by 122
- 2025Agent Lightning: Train ANY AI Agents with Reinforcement Learningcited by 94
- 2025In-the-Flow Agentic System Optimization for Effective Planning and Tool Usecited by 67
- 2025Training Proactive and Personalized LLM Agentscited by 42
- 2025Distilling LLM Agent into Small Models with Retrieval and Code Toolscited by 41
- 2026Evolving Agents in the Dark: Retrospective Harness Optimization via Self-Preferencecited by 17
- 2026Learning Personalized Agents from Human Feedbackcited by 16
- 2026When Tools Fail: Benchmarking Dynamic Replanning and Anomaly Recovery in LLM Agentscited by 15
- 2023Learning to (Learn at Test Time)cited by 12
- 2026Evo-Harness: Context-to-Harness Skill Compilation for Self-Evolving Agentscited by 12
- 2026Step-wise Rubric Rewards for LLM Reasoningcited by 11
- 2026Reproducing, Analyzing, and Detecting Reward Hacking in Rubric-Based Reinforcement Learningcited by 9
- 2026HarnessDev: Can LLMs Create and Evolve Their Own Agent Harness?cited by 7
- 2026When Simulation Lies: A Sim-to-Real Benchmark and Domain-Randomized RL Recipe for Tool-Use Agentscited by 5
- 2026HarnessOpt-Bench: Evaluating LLMs at Harness Optimizationcited by 5
- 2026AI4AI at Test-Time: Strong-to-Weak Capability Transfer via Harnessescited by 5
- 2026Reinforcement Learning for LLM-based Multi-Agent Systems through Orchestration Tracescited by 4
- 2026LiteGUI: Distilling Compact GUI Agents with Reinforcement Learningcited by 4
- 2026Learning to Control LLM Agent Harnesses with Offline Reinforcement Learningcited by 3
- 2026LLM Judges Can Be Too Generous When There Is No Reference Answercited by 3
- 2026Beyond Prompts: Measuring and Optimizing LLM Tool-Agent Harnessescited by 3
- 2026GAUGE: When Not to Trust LLM-as-a-Judge in User-Simulated Evaluation of Task-Oriented Agentscited by 3
- 2026Harness-Zero: Harness Distillation via Agent-as-Harnesscited by 3
- 2026UXBench: Benchmarking User Experience in AI Assistantscited by 2
- 2026Workflow-Localized Mechanism Learning: Attribution-Guided Repair and Knowledge Reuse for Structured Agent Skillscited by 2
- 2026CodeGrep: An RL-Trained Retrieval Agent for LLM Coding Agentscited by 2
- 2025PromptBridge: Cross-Model Prompt Transfer for Large Language Modelscited by 1
- 2026EMPATH: A Multilingual Auditor-Judge Benchmark for Safety Evaluation of Emotional-Support Chatbotscited by 1
- 2026PERSONAJUDGE: Simulating Individual Human Preference Judgments with Evaluator-Specific Demonstration Datacited by 1
- 2026From Profiling to Synthesis: Benchmarking Implicit Behavioral Alignment in Personalized LLM Agentscited by 1
- 2026DiDPO: Diff-in-Diff Policy Optimization for Coding Agent Trainingcited by 1
- 2026Judge, Retrieve, or Abstain: Uncertainty-Guarded LLM Judging with Provable Risk Guaranteescited by 1
- 2026Design-to-Plan: A Large Language Model-Based Multi-Agent Framework for Manufacturing Process Planning from 3D CAD Models and 2D Engineering Drawingscited by 1
- 2026HarnessBandit: Joint Learnability-Transferability Scheduling for Multi-Harness Agentic Reinforcement Learningcited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026The Confidence Dichotomy: Analyzing and Mitigating Miscalibration in Tool-Use Agents
- 2026MARGIN: Runtime Confidence Calibration for Multi-Agent Foundation Model Coordination
- 2026TRACE: A Self-Evolving Skill Bank for Consistent, Limit-Aware LLM Agents
- 2026LiveMACE: Process-Aware Evaluation of LLM Agent Capabilities in Evolving Markets
- 2026The Double Measurement Confound in Agent Benchmarks: De-Scaffolding, Ground-Truth Scoring, and Reliability Beyond the Mean
- 2026A Contract-Centered Architecture for Scalable and Manageable Agentic Runtimes
- 2026CAVEAT: Towards Robust Computer-Use Agents in Incentive-Misaligned Environments
- 2026EconSkills: Studying Skill Transfer and Retrieval for Web Agents on Live Economic Data