Physical Kernel: Structured Visual Latents for Dark Manipulation
Not on the map. Why?
Abstract (arXiv)
We study dark manipulation: after a brief lit Write encodes z0 = Enc(rgb), a policy pi(z) and open-loop dynamics f(z,a) complete contact-rich skills without further pixels (dark_f). On ManiSkill StackCube (n=160; seed packs 0/1000), dark_f attains 68.1% stacked on the five-rung chain (near_A -> grasped -> lifted -> on_B -> stacked), compared with 35.6% for per-step lit_reenc and 0% for freeze/encode_black. On a shared Write->HOLD protocol (n=40), occlusion and camera-aligned GT contact-neighbor masks drive lit lift from 43% to 0%, while dark_f holds 82.5%; shuffling actions inflates dynamics MSE by ~9.4x; write-time appearance shifts break encoding (night: 0% stacked), yet the same shifts during HOLD leave dark_f lift unchanged; Write length Tw is flat once the stop phase is reached, while earlier stops and write-time blur/JPEG sharply cut stacked. A dedicated pi_write reaches 35% vision-budget stacked (n=80); matched Dreamer-style/pixel nulls without privileged geom stay at 0%. Privileged state-RSSM MPC reaches ~35% stacked with 9D dark observations -- a stronger-observation null, not a matched visual baseline.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 12 of its 15 references that are in the map. First posted on arXiv 4 Sep 2026.
Builds on
12 of 15 references.
References · 12
- 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,838
- 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
- 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,568
- 2018Learning Latent Dynamics for Planning from Pixelscited by 1,264
- 2023Mastering Diverse Domains through World Modelscited by 1,253
- 2023Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecturecited by 1,039
- 2024Revisiting Feature Prediction for Learning Visual Representations from Videocited by 531
- 2017Time-Contrastive Networks: Self-Supervised Learning from Videocited by 459
- 2021Implicit Behavioral Cloningcited by 452
- 2023ManiSkill2: A Unified Benchmark for Generalizable Manipulation Skillscited by 327
- 2024ManiSkill3: GPU Parallelized Robotics Simulation and Rendering for Generalizable Embodied AIcited by 229
- 2019PHYRE: A New Benchmark for Physical Reasoningcited by 137
Built on by
Not cited yet.