# From Pixels to CSI: Distilling Latent Dynamics For Efficient Wireless Resource Management

arXiv: https://arxiv.org/abs/2506.16216 · HTML: https://mlsanity.com/paper/2506.16216/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 20 references that are in the map.
- First posted on arXiv 19 Jun 2025.

## Builds on (most cited first, 10 of 10)

- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Learning Latent Dynamics for Planning from Pixels](https://mlsanity.com/paper/1811.04551/) (2018)
- [Mastering Diverse Domains through World Models](https://mlsanity.com/paper/2301.04104/) (2023)
- [Mastering Atari with Discrete World Models](https://mlsanity.com/paper/2010.02193/) (2020)
- [Open X-Embodiment: Robotic Learning Datasets and RT-X Models](https://mlsanity.com/paper/2310.08864/) (2023)
- [DayDreamer: World Models for Physical Robot Learning](https://mlsanity.com/paper/2206.14176/) (2022)
- [Sionna: An Open-Source Library for Next-Generation Physical Layer Research](https://mlsanity.com/paper/2203.11854/) (2022)
- [Time-Series JEPA for Predictive Remote Control under Capacity-Limited Networks](https://mlsanity.com/paper/2406.04853/) (2024)
- [Learning Latent Wireless Dynamics from Channel State Information](https://mlsanity.com/paper/2409.10045/) (2024)
- [Task-Oriented Prediction and Communication Co-Design for Haptic Communications](https://mlsanity.com/paper/2302.11064/) (2023)

## Built on by (most cited first, 2 of 2)

- [Hierarchical JEPA Meets Predictive Remote Control in Beyond 5G Networks](https://mlsanity.com/paper/2602.07000/) (2026)
- [Coupled Control and Wireless World Models for Resilient Remote Robotic Control](https://mlsanity.com/paper/2609.04851/) (2026)

## Abstract (arXiv)

In this work, we aim to optimize the radio resource management of a communication system between a remote controller and its device, whose state is represented through image frames, without compromising the performance of the control task. We propose a novel machine learning (ML) technique to jointly model and predict the dynamics of the control system as well as the wireless propagation environment in latent space. Our method leverages two coupled joint-embedding predictive architectures (JEPAs): a control JEPA models the control dynamics and guides the predictions of a wireless JEPA, which captures the dynamics of the device's channel state information (CSI) through cross-modal conditioning. We then train a deep reinforcement learning (RL) algorithm to derive a control policy from latent control dynamics and a power predictor to estimate scheduling intervals with favorable channel conditions based on latent CSI representations. As such, the controller minimizes the usage of radio resources by utilizing the coupled JEPA networks to imagine the device's trajectory in latent space. We present simulation results on synthetic multimodal data and show that our proposed approach reduces transmit power by over 50% while maintaining control performance comparable to baseline methods that do not account for wireless optimization.