mlsanityData to 14 Oct 2026

VERA: Verifiable Feasibility Representations with Counterfactual Credit for Constrained Multi-Agent Control

arXiv 2609.38889 ↗First posted 30 Sep 2026cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Constrained multi-agent control requires more than predicting rewarding actions: an action can cease to be executable as contact windows, shared capacity, and deadlines change. We introduce VERA, a centralized-training, decentralized-execution framework that separates feasibility estimation from credit assignment. Each actor predicts a five-dimensional verifiable feasibility representation (VFR). After an action is proposed, exact action-conditioned margins available only during training supervise that representation, while a counterfactual group-relative advantage (CGRA) ranks candidate representation-action pairs. Execution uses one actor pass and no privileged state. In a dynamic space-air-ground integrated network (SAGIN), VERA obtains 55.33% +/- 3.60% success with 0.45% +/- 0.81% coverage violation, within 1.33 percentage points of a privileged-mask reference. With rewards matched over ten paired seeds, VERA improves success over the strongest baseline by 8.74 percentage points (p=0.023) and reduces violation by 52.19 percentage points (p=5.7e-8). A ten-seed 4-by-2 factorial attributes a 14.16-16.48 percentage-point gain to CGRA across handcrafted, learned, random, and latent representations; evaluation on seven unseen topologies preserves a 24.33-30.02 percentage-point advantage over multi-agent proximal policy optimization. From 10 to 40 users, success remains 50.1-53.8%, and VFR adds only 0.026 ms to a central processing unit (CPU) actor step. Cross-domain tests further identify the governing condition: counterfactual credit succeeds when candidate scores respect shared constraints and fails under incompatible reward geometries. These results establish action-conditioned feasibility as an auditable training interface and counterfactual credit as a geometry-dependent optimization mechanism.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 42 of its 54 references that are in the map. First posted on arXiv 30 Sep 2026.

Builds on

42 of 54 references.

References · 42

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  4. 2023Let's Verify Step by Stepcited by 3,789
  5. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 2,026
  6. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,558
  7. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,546
  8. 2025Understanding R1-Zero-Like Training: A Critical Perspectivecited by 1,345
  9. 2020Concept Bottleneck Modelscited by 1,032
  10. 2017Constrained Policy Optimizationcited by 983
  11. 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 942
  12. 2017Counterfactual Multi-Agent Policy Gradientscited by 751
30 more
  1. 2023Safe RLHF: Safe Reinforcement Learning from Human Feedbackcited by 656
  2. 2020Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 485
  3. 2021Trust Region Policy Optimisation in Multi-Agent Reinforcement Learningcited by 186
  4. 2020FACMAC: Factored Multi-Agent Centralised Policy Gradientscited by 178
  5. 2022Multi-Agent Reinforcement Learning is a Sequence Modeling Problemcited by 155
  6. 2023Safety-Gymnasium: A Unified Safe Reinforcement Learning Benchmarkcited by 143
  7. 2022SMACv2: An Improved Benchmark for Cooperative Multi-Agent Reinforcement Learningcited by 122
  8. 2020Invariant Causal Prediction for Block MDPscited by 105
  9. 2023Probabilistic Concept Bottleneck Modelscited by 101
  10. 2019Optimal Wireless Resource Allocation with Random Edge Graph Neural Networkscited by 90
  11. 2023OmniSafe: An Infrastructure for Accelerating Safe Reinforcement Learning Researchcited by 80
  12. 2022Saute RL: Almost Surely Safe Reinforcement Learning Using State Augmentationcited by 69
  13. 2023Heterogeneous-Agent Reinforcement Learningcited by 69
  14. 2022VMAS: A Vectorized Multi-Agent Simulator for Collective Robot Learningcited by 68
  15. 2022Causal Dynamics Learning for Task-Independent State Abstractioncited by 65
  16. 2022Constrained Update Projection Approach to Safe Policy Optimizationcited by 58
  17. 2022Bisimulation Makes Analogies in Goal-Conditioned Reinforcement Learningcited by 38
  18. 2022Revisiting Some Common Practices in Cooperative Multi-Agent Reinforcement Learningcited by 32
  19. 2022On the Generalization of Representations in Reinforcement Learningcited by 28
  20. 2019A Graph Neural Network Approach for Scalable Wireless Power Controlcited by 26
  21. 2024Interpretable Concept Bottlenecks to Align Reinforcement Learning Agentscited by 23
  22. 2023Maximum Entropy Heterogeneous-Agent Reinforcement Learningcited by 19
  23. 2023Provably Efficient Generalized Lagrangian Policy Optimization for Safe Multi-Agent Reinforcement Learningcited by 14
  24. 2024Off-Policy Primal-Dual Safe Reinforcement Learningcited by 14
  25. 2023Dynamic Routing for Integrated Satellite-Terrestrial Networks: A Constrained Multi-Agent Reinforcement Learning Approachcited by 9
  26. 2018Decentralized Computation Offloading for Multi-User Mobile Edge Computing: A Deep Reinforcement Learning Approachcited by 8
  27. 2024Joint Offloading and Resource Allocation for Hybrid Cloud and Edge Computing in SAGINs: A Decision Assisted Hybrid Action Space Deep Reinforcement Learning Approachcited by 5
  28. 2024On the Interplay of Artificial Intelligence and Space-Air-Ground Integrated Networks: A Surveycited by 5
  29. 2024Feasibility Consistent Representation Learning for Safe Reinforcement Learningcited by 3
  30. 2024Cost-Efficient Computation Offloading in SAGIN: A Deep Reinforcement Learning and Perception-Aided Approachcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026ASCT: Attentive Search over Counterfactual Trees for Credit Assignment in Agentic Reinforcement Learning
  2. 2026BiPACE: Bisimulation-Guided Policy Optimization with Action Counterfactual Estimation for LLM Agents
  3. 2026Coverage, Not Targeting: A Structural Regime in Multi-Turn Agent Credit Assignment
  4. 2026How Much, Then Where: Credit-Conserving Action-to-Token Allocation for Multi-Turn Agent Reinforcement Learning
  5. 2026VICT: Verifier-Instrumented Credit Tracing for Long-Horizon LLM Agent Reinforcement Learning
  6. 2026A Self-Evolving Default Action for Cooperative Tasks with Continuous Action Space
  7. 2026Teach the Magnitude, Not the Direction: Verifier-Bounded Credit Assignment for Multi-Turn Multi-step LLM Agents
  8. 2026VETTA: Coordinating Turn- and Token-Level Credit Assignment for Multi-Turn LLM Agents