Operator-on-F complements value-equivalence: a planning-time diagnostic for latent world models
On the map: World models → Latent dynamics models and model-based RL
Abstract (arXiv)
World-model evaluation for model-based reinforcement learning typically asks whether the learned model predicts reward and value well, which can leave planning-relevant errors in the model's latent rollouts unmeasured. We introduce a complementary diagnostic, operator-on-F, that compares a model's k-step latent pushforward to the environment's on an observable subset F, using the model's own predictor. On a TD-MPC2 size sweep over cheetah-run, reward-prediction error stays within [0.028, 0.091] for every model size - only about 3x variation - so an unnormalized reward-fit check has narrow resolution to distinguish them; the (unnormalized) Bellman residual and reward error themselves have weak relationships with return (Spearman -0.10 and -0.30). Operator error spans 0.28 to 2.62 over the same sizes. At 317M the operator error is 2.62 - an order of magnitude above the 0.28-0.36 cluster - and the planning return collapses to 0.9, while reward-prediction error (0.091) is the highest of the five but stays within the same small [0.028, 0.091] range as the rest of the sweep. The rank correlation between operator error and return loss is -0.90 (anchor-bootstrap 95% CI [-0.90, -0.70] at n=5 sizes; leave-one-out removal of any single size leaves it at -0.80 or stronger). The operator also returns informative, architecture-discriminating estimates in a cross-architecture comparison between TD-MPC2 and a pure-SSL latent world model. The operator diagnostic complements value-equivalence rather than replacing it.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Latent dynamics models and model-based RL, one of 11 directions in World models. Builds on 10 of its 15 references that are in the map. First posted on arXiv 5 Jul 2026.
Builds on
10 of 15 references.
Closest ancestors · 2
Same direction, newest first
- 2023TD-MPC2: Scalable, Robust World Models for Continuous Control25 Oct
- 2023Mastering Diverse Domains through World Models10 Jan
Other references · 8
- 2016Understanding intermediate layers using linear classifier probescited by 1,279
- 2019Designing and Interpreting Probes with Control Taskscited by 608
- 2016Linear predictors for nonlinear dynamical systems: Koopman operator meets model predictive controlcited by 134
- 2016The Predictron: End-To-End Learning and Planningcited by 92
- 2020The Value Equivalence Principle for Model-Based Reinforcement Learningcited by 88
- 2021Proper Value Equivalencecited by 33
- 2023Task-Oriented Koopman-Based Control with Contrastive Encodercited by 8
- 2025Koopman Invariants as Drivers of Emergent Time-Series Clustering in Joint-Embedding Predictive Architecturescited by 3
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2026Inverting the Bellman Equation: From Q-Values to World Models
- 2026Scaling World-Model Reinforcement Learning Through Diffusion Policy Optimization
- 2026A Control Theory of Predictability in Latent World Models
- 2026On Training in Imagination
- 2026When Low Prediction Error Misleads Planning: Diagnosing Representation, Dynamics, and Decision Failures in Latent World Models
- 2026When Errors Can Be Beneficial: A Categorization of Imperfect Rewards for Policy Gradient
- 2026Where the Model Changes Its Mind: Hindsight-Divergence Localization for Efficient Reinforcement Learning with Verifiable Rewards
- 2026Reward Models Are Secretly Value Functions: Temporally Coherent Reward Modeling