# JoyAI-Sim: A Simulation-Enabled Interconversion Toolchain for the Embodied Data Pyramid

arXiv: https://arxiv.org/abs/2606.16776 · HTML: https://mlsanity.com/paper/2606.16776/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Jun 2026.

## Built on by (most cited first, 3 of 3)

- [HaWMPO: Hallucination-Aware World Model-based Policy Optimization for Generalist Robot Policy](https://mlsanity.com/paper/2609.09941/) (2026)
- [WLA3: World Latent Action Modeling for Semantics, Dynamics, and Kinematics](https://mlsanity.com/paper/2609.15870/) (2026)
- [LAPT-Scale: Scaling Embodied Policies with Latent Action Pre-Training on Human Videos](https://mlsanity.com/paper/2610.13732/) (2026)

## Abstract (arXiv)

Generalist robot policies require trustworthy evaluation and robot-usable training data, but both are difficult to scale with physical robots alone. Real-robot trials and demonstrations remain the most faithful source of deployment signals, yet they are slow, costly, and hard to reproduce. We present JoyAI-Sim, a simulation-enabled interconversion toolchain for human-robot aligned model evaluation and data generation, denoted as Robot rightleftharpoons Simulation rightleftharpoons Human. On the one hand, the Robot rightarrow Simulation rightarrow Human pathway supports human-robot aligned model evaluation by reconstructing real-robot tabletop organization tasks as calibrated digital twins for scalable evaluation, while using human embodied feedback to inspect and refine the naturalness of simulated motions. On the other hand, the Human rightarrow Simulation rightarrow Robot pathway supports human-robot aligned data generation: it lifts ego-centric human demonstrations into simulation, checks them under robot physical constraints, and converts them into robot-centered trajectories, annotations, and visual observations. Together, these pathways use the JoySim simulator as both a scalable evaluation layer and a physical consistency filter for robot data generation. We further package the core reconstruction, simulation, rendering, and realism-augmentation modules as cloud services on JD Cloud, turning the system into a reusable and scalable infrastructure for robot data generation and model evaluation.