FactorJEPA: Factorizing Monolithic Futures into Layout-Agent-Interaction Channels for Crowded and Chaotic Global South Urban Worlds
On the map: World models → Video-generation world models and simulators
Abstract (arXiv)
World models have attracted significant attention for their ability to capture and predict the structure and dynamics of the physical world. In this emerging landscape, Joint Embedding Predictive Architectures (JEPA) offer a particularly compelling direction. We study a largely unexplored regime: populous, crowded, and chaotic Global South urban environments, which we call DENSEWORLD. Unlike the lower-density, lane-structured settings that dominate existing evaluations, these scenes exhibit soft spatial boundaries, extreme agent heterogeneity, persistent occlusion, and rapid social negotiation under mixed traffic. We introduce the first large-scale dataset for this regime: 1,000 hours of drive-through, walk-through, and aerial video across 22 cities. Existing JEPA formulations struggle to preserve dense interaction dynamics under heterogeneity and partial observability. We introduce FactorJEPA, which makes world structure a first-class predictive primitive. Rather than encoding the future in a monolithic latent, it composes layout, entities, and interactions, using a visibility gate and separated subspaces to preserve partially observed agents and discourage cross-factor shortcuts. FactorJEPA improves (i) future-latent accuracy (Future-frame L1), (ii) intervention-sensitive prediction (Causal L1), and (iii) robustness to reduced visual evidence (Mask-ratio slope), while exposing (iv) a reproducible motion-information trade-off (Motion cosine). Method rankings replicate across 2B and 1B V-JEPA 2.1 backbones, with rho = 0.895 to 0.978. We publicly release the DENSEWORLD-115k dataset (https://huggingface.co/datasets/anonymousML123/denseworld-115k) and the surgery-trained FactorJEPA checkpoints (https://huggingface.co/datasets/anonymousML123/factorjepa-outputs/tree/main/outputs/full/vjepa_2_1_vitg_1B/train/m09c_surgery_3stage_DI_diheavy_encoder).
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Video-generation world models and simulators, one of 11 directions in World models. Builds on 5 of its 14 references that are in the map. First posted on arXiv 2 Aug 2026.
Builds on
5 of 14 references.
References · 5
- 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
- 2018BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learningcited by 1,282
- 2025Cosmos World Foundation Model Platform for Physical AIcited by 833
- 2018IDD: A Dataset for Exploring Problems of Autonomous Navigation in Unconstrained Environmentscited by 163
- 2022IDD-3D: Indian Driving Dataset for 3D Unstructured Road Scenescited by 11
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Hi-FLoop: Hierarchical State-Feedback Loops for Multi-Timescale World Modeling
- 2026Omni-WorldBench: Towards a Comprehensive Interaction-Centric Evaluation for World Models
- 2026WALL-SS: Scaling Long-horizon World Models via Next-Scale Autoregression
- 2026Population-Scalable Multi-Agent World Modeling
- 2026Oneira: From Open-Ended Generation to Open-World Interaction in Video World Models
- 2026LRC-JEPA: Disentangling Dynamics and Residual Context for Efficient World Models
- 2025Learning 3D Persistent Embodied World Models
- 2026VisualPatchWorld: Code World Models as Latent Structured Representations for Planning