mlsanityData to 7 Oct 2026

Operator-on-F complements value-equivalence: a planning-time diagnostic for latent world models

arXiv 2607.04464 ↗First posted 5 Jul 2026cs.LG cs.AIcited by 2

On the map: World models → Latent dynamics models and model-based RL

Abstract (arXiv)

World-model evaluation for model-based reinforcement learning typically asks whether the learned model predicts reward and value well, which can leave planning-relevant errors in the model's latent rollouts unmeasured. We introduce a complementary diagnostic, operator-on-F, that compares a model's k-step latent pushforward to the environment's on an observable subset F, using the model's own predictor. On a TD-MPC2 size sweep over cheetah-run, reward-prediction error stays within [0.028, 0.091] for every model size - only about 3x variation - so an unnormalized reward-fit check has narrow resolution to distinguish them; the (unnormalized) Bellman residual and reward error themselves have weak relationships with return (Spearman -0.10 and -0.30). Operator error spans 0.28 to 2.62 over the same sizes. At 317M the operator error is 2.62 - an order of magnitude above the 0.28-0.36 cluster - and the planning return collapses to 0.9, while reward-prediction error (0.091) is the highest of the five but stays within the same small [0.028, 0.091] range as the rest of the sweep. The rank correlation between operator error and return loss is -0.90 (anchor-bootstrap 95% CI [-0.90, -0.70] at n=5 sizes; leave-one-out removal of any single size leaves it at -0.80 or stronger). The operator also returns informative, architecture-discriminating estimates in a cross-architecture comparison between TD-MPC2 and a pure-SSL latent world model. The operator diagnostic complements value-equivalence rather than replacing it.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Latent dynamics models and model-based RL, one of 11 directions in World models. Builds on 10 of its 15 references that are in the map. First posted on arXiv 5 Jul 2026.

Builds on

10 of 15 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023TD-MPC2: Scalable, Robust World Models for Continuous Control25 Oct
  2. 2023Mastering Diverse Domains through World Models10 Jan

Other references · 8

  1. 2016Understanding intermediate layers using linear classifier probescited by 1,279
  2. 2019Designing and Interpreting Probes with Control Taskscited by 608
  3. 2016Linear predictors for nonlinear dynamical systems: Koopman operator meets model predictive controlcited by 134
  4. 2016The Predictron: End-To-End Learning and Planningcited by 92
  5. 2020The Value Equivalence Principle for Model-Based Reinforcement Learningcited by 88
  6. 2021Proper Value Equivalencecited by 33
  7. 2023Task-Oriented Koopman-Based Control with Contrastive Encodercited by 8
  8. 2025Koopman Invariants as Drivers of Emergent Time-Series Clustering in Joint-Embedding Predictive Architecturescited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026Inverting the Bellman Equation: From Q-Values to World Models
  2. 2026Scaling World-Model Reinforcement Learning Through Diffusion Policy Optimization
  3. 2026A Control Theory of Predictability in Latent World Models
  4. 2026On Training in Imagination
  5. 2026When Low Prediction Error Misleads Planning: Diagnosing Representation, Dynamics, and Decision Failures in Latent World Models
  6. 2026When Errors Can Be Beneficial: A Categorization of Imperfect Rewards for Policy Gradient
  7. 2026Where the Model Changes Its Mind: Hindsight-Divergence Localization for Efficient Reinforcement Learning with Verifiable Rewards
  8. 2026Reward Models Are Secretly Value Functions: Temporally Coherent Reward Modeling