mlsanityData to 7 Oct 2026

A Context Engineering Framework for Improving Enterprise AI Agents based on Digital-Twin MDP

arXiv 2603.22083 ↗First posted 23 Mar 2026cs.AIcited by 0

On the map: LLM agents and tool use → Reinforcement learning for LLM agents

Abstract (arXiv)

Despite rapid progress in AI agents for enterprise automation and decision-making, their real-world deployment and further performance gains remain constrained by limited data quality and quantity, complex real-world reasoning demands, difficulties with self-play, and the lack of reliable feedback signals. To address these challenges, we propose a lightweight, model-agnostic framework for improving LLM-based enterprise agents via offline reinforcement learning (RL). The proposed Context Engineering via DT-MDP (DT-MDP-CE) framework comprises three key components: (1) A Digital-Twin Markov Decision Process (DT-MDP), which abstracts the agent's reasoning behavior as a finite MDP; (2) A robust contrastive inverse RL, which, armed with the DT-MDP, to efficiently estimate a well-founded reward function and induces policies from mixed-quality offline trajectories; and (3) RL-guided context engineering, which uses the policy obtained from the integrated process of (1) and (2), to improve the agent's decision-making behavior. As a case study, we apply the framework to a representative task in the enterprise-oriented domain of IT automation. Extensive experimental results demonstrate consistent and significant improvements over baseline agents across a wide range of evaluation settings, suggesting that the framework can generalize to other agents sharing similar characteristics in enterprise environments.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reinforcement learning for LLM agents, one of 16 directions in LLM agents and tool use. Builds on 38 of its 70 references that are in the map. First posted on arXiv 23 Mar 2026.

Builds on

38 of 70 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025Agent Lightning: Train ANY AI Agents with Reinforcement Learning5 Aug

Other references · 37

  1. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  2. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  3. 2017Deep reinforcement learning from human preferencescited by 3,846
  4. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
  5. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  6. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
  7. 2016Generative Adversarial Imitation Learningcited by 1,576
  8. 2022Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questionscited by 911
  9. 2024Understanding the planning of LLM agents: A surveycited by 414
  10. 2022RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learningcited by 361
  11. 2018A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progresscited by 294
  12. 2019Extrapolating Beyond Suboptimal Demonstrations via Inverse Reinforcement Learning from Observationscited by 256
25 more
  1. 2023How Abilities in Large Language Models are Affected by Supervised Fine-tuning Data Compositioncited by 222
  2. 2018Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learningcited by 203
  3. 2019Imitation Learning via Off-Policy Distribution Matchingcited by 182
  4. 2025A Survey of Reinforcement Learning for Large Reasoning Modelscited by 175
  5. 2025From LLM Reasoning to Autonomous AI Agents: A Comprehensive Reviewcited by 146
  6. 2025Evaluation and Benchmarking of LLM Agents: A Surveycited by 145
  7. 2020Hyperparameter Selection for Offline Reinforcement Learningcited by 113
  8. 2019Imitation Learning from Imperfect Demonstrationcited by 112
  9. 2022A Review of Off-Policy Evaluation in Reinforcement Learningcited by 92
  10. 2021Benchmarks for Deep Off-Policy Evaluationcited by 89
  11. 2022Discriminator-Weighted Offline Imitation Learning from Suboptimal Demonstrationscited by 89
  12. 2021d3rlpy: An Offline Deep Reinforcement Learning Librarycited by 84
  13. 2017Continuous State-Space Models for Optimal Sepsis Treatment - a Deep Reinforcement Learning Approachcited by 68
  14. 2021Scalable Bayesian Inverse Reinforcement Learningcited by 56
  15. 2025ITBench: Evaluating AI Agents across Diverse Real-World IT Automation Taskscited by 48
  16. 2021TRAIL: Near-Optimal Imitation Learning with Suboptimal Datacited by 39
  17. 2025RL Tango: Reinforcing Generator and Verifier Together for Language Reasoningcited by 39
  18. 2024Rethinking Data Selection for Supervised Fine-Tuningcited by 35
  19. 2019Truly Batch Apprenticeship Learning with Deep Successor Featurescited by 27
  20. 2020Sequential Anomaly Detection using Inverse Reinforcement Learningcited by 19
  21. 2023Formally Specifying the High-Level Behavior of LLM-Based Agentscited by 15
  22. 2024Offline Imitation Learning with Model-based Reverse Augmentationcited by 8
  23. 2025Enhancing Large Language Model Reasoning with Reward Models: An Analytical Surveycited by 7
  24. 2026Think Locally, Explain Globally: Graph-Guided LLM Investigations via Local Reasoning and Belief Propagationcited by 3
  25. 2025Understanding Inverse Reinforcement Learning under Overparameterization: Non-Asymptotic Analysis and Global Optimalitycited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Towards an Information Theoretic Framework of Context-Based Offline Meta-Reinforcement Learning
  2. 2026Beyond Reward Engineering: A Data Recipe for Long-Context Reinforcement Learning
  3. 2025Advancing Multi-Agent Systems Through Model Context Protocol: Architecture, Implementation, and Applications
  4. 2025The Landscape of Agentic Reinforcement Learning for LLMs: A Survey
  5. 2025Agent-as-Tool: A Study on the Hierarchical Decision Making with Reinforcement Learning
  6. 2026Unified Context Evolution for LLM Agents
  7. 2026KARL: Knowledge Agents via Reinforcement Learning
  8. 2025Exploring the Potential of Offline RL for Reasoning in LLMs: A Preliminary Study