FOLIAGE: Towards Physical Intelligence World Models Via Unbounded Surface Evolution
On the map: World models → Latent predictive world models
Abstract (arXiv)
Many physical systems do not merely move or deform; they grow, adding material and changing the geometry that a world model must represent. Existing world models are typically optimized for pixel prediction, reward prediction, or fixed-support physical dynamics, leaving open how to model systems whose underlying physical support expands over time and whose future morphology depends on hidden material response. We introduce FOLIAGE, a geometry-centered latent world model for growing surfaces. Within a fixed state budget, FOLIAGE represents mature regions as a compact scaffold while allocating higher-resolution state to regions predicted to drive near-future growth. This focuses representation and computation where new material and geometric change occur while retaining compact global context. FOLIAGE further separates observation, action, and privileged physics: heterogeneous RGB, point-cloud, and mesh observations are fused into a deployable geometric state; material controls condition the latent dynamics; and hidden physical energies guide training but are not required at deployment. To evaluate this setting, we introduce SURF-GARDEN and SURF-BENCH, providing controlled counterfactual branches, dense cross-modal correspondences, hidden physical signals, and stress tests for growing-geometry state learning. FOLIAGE reduces inverse-material error by ≈40% and 5-step mesh forecasting Chamfer error by ≈30% relative to strong baselines, while improving cross-modal retrieval by +14 mAP points. Stress tests show graceful degradation under sensor loss and correspondence corruption. On temporal 3D plant scans, FOLIAGE also improves passive future-geometry forecasting, while transfer experiments show that the learned geometry-centered state remains useful beyond the simulator.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Latent predictive world models, one of 11 directions in World models. Builds on 45 of its 72 references that are in the map. First posted on arXiv 29 May 2025.
Builds on
45 of 72 references.
Closest ancestors · 1
Same direction, newest first
Other references · 44
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2015Distilling the Knowledge in a Neural Networkcited by 10,324
- 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
- 2016Layer Normalizationcited by 5,669
- 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,401
- 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,057
- 2017Improved Regularization of Convolutional Neural Networks with Cutoutcited by 1,628
- 2021Is Space-Time Attention All You Need for Video Understanding?cited by 1,620
- 2018World Modelscited by 1,513
- 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,498
- 2023Mastering Diverse Domains through World Modelscited by 1,253
- 2021Perceiver: General Perception with Iterative Attentioncited by 1,104
32 more
- 2021Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learningcited by 998
- 2020Learning to Simulate Complex Physics with Graph Networkscited by 880
- 2020Learning Mesh-Based Simulation with Graph Networkscited by 868
- 2023VideoMAE V2: Scaling Video Masked Autoencoders with Dual Maskingcited by 528
- 2021Physics-Informed Neural Operator for Learning Partial Differential Equationscited by 527
- 2021Habitat 2.0: Training Home Assistants to Rearrange their Habitatcited by 511
- 2021PointCLIP: Point Cloud Understanding by CLIPcited by 463
- 2022PointNeXt: Revisiting PointNet++ with Improved Training and Scaling Strategiescited by 436
- 2024VideoMamba: State Space Model for Efficient Video Understandingcited by 328
- 2023MeshGPT: Generating Triangle Meshes with Decoder-Only Transformerscited by 259
- 2019DiffTaichi: Differentiable Programming for Physical Simulationcited by 258
- 2021GRAND: Graph Neural Diffusioncited by 258
- 2023ULIP-2: Towards Scalable Multimodal Pre-training for 3D Understandingcited by 202
- 2022CrossPoint: Self-Supervised Cross-Modal Contrastive Learning for 3D Point Cloud Understandingcited by 198
- 2018Sim-to-Real via Sim-to-Sim: Data-efficient Robotic Grasping via Randomized-to-Canonical Adaptation Networkscited by 174
- 2020DiffusionNet: Discretization Agnostic Learning on Surfacescited by 164
- 2022CLIP2Point: Transfer CLIP to Point Cloud Classification with Image-Depth Pre-trainingcited by 160
- 2020Deep Geometric Functional Maps: Robust Feature Learning for Shape Correspondencecited by 125
- 2018ChainQueen: A Real-Time Differentiable Physical Simulator for Soft Roboticscited by 98
- 2022SVFormer: Semi-supervised Video Transformer for Action Recognitioncited by 68
- 2021DiffCloth: Differentiable Cloth Simulation with Dry Frictional Contactcited by 65
- 2021DiffPD: Differentiable Projective Dynamicscited by 62
- 2022CaDeX: Learning Canonical Deformation Coordinate Space for Dynamic Surface Representation via Neural Homeomorphismcited by 57
- 2023PiMAE: Point Cloud and Image Interactive Masked Autoencoders for 3D Object Detectioncited by 53
- 2024Motion2VecSets: 4D Latent Vector Set Diffusion for Non-rigid Shape Reconstruction and Trackingcited by 29
- 2024Spectral Meets Spatial: Harmonising 3D Shape Matching and Interpolationcited by 26
- 2024E(n) Equivariant Topological Neural Networkscited by 22
- 2022GrowliFlower: An image time series dataset for GROWth analysis of cauLIFLOWERcited by 15
- 2022G-MSM: Unsupervised Multi-Shape Matching with Graph-based Affinity Priorscited by 8
- 2023Adaptive Topological Feature via Persistent Homology: Filtration Learning for Point Cloudscited by 8
- 2023NeuralClothSim: Neural Deformation Fields Meet the Thin Shell Theorycited by 7
- 2024Bayesian Differentiable Physics for Cloth Digitalizationcited by 2
Built on by
Citing papers per year: 2025: 1; 2026: 1.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2026InternW0: A Foundational Physical World Model for Efficient Real-World Interactions
- 2026Emergent Semantic Representations in World Models through Physical Interaction without Linguistic Supervision
- 2026The Trinity of Consistency as a Defining Principle for General World Models
- 2026PhiZero: A World Model Built Around Physical Language
- 2026OmniFysics: Towards Physical Intelligence Evolution via Omni-Modal Signal Processing and Network Optimization
- 2026WorldCast: Exploring Intelligent Visual Generation in World State Prediction
- 2026Physically Native World Models: A Hamiltonian Perspective on Generative World Modeling
- 2026OneWorld: Learning Consistent Physics Across Actions in World Models