# Isaac Lab: A GPU-Accelerated Simulation Framework for Multi-Modal Robot Learning

arXiv: https://arxiv.org/abs/2511.04831 · HTML: https://mlsanity.com/paper/2511.04831/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Nov 2025.

## Built on by (most cited first, 24 of 24)

- [Agentic World Modeling: Foundations, Capabilities, Laws, and Beyond](https://mlsanity.com/paper/2604.22748/) (2026)
- [Reconstruction by Generation: 3D Multi-Object Scene Reconstruction from Sparse Observations](https://mlsanity.com/paper/2604.27106/) (2026)
- [Phys4D: Fine-Grained Physics-Consistent 4D Modeling from Video Diffusion](https://mlsanity.com/paper/2603.03485/) (2026)
- [RobotWorld: Benchmarking Multimodal Agents for Robot Use Across Diverse Tasks and Embodiments](https://mlsanity.com/paper/2610.10409/) (2026)
- [Residual Reinforcement Learning for Waste-Container Lifting Using Large-Scale Cranes with Underactuated Tools](https://mlsanity.com/paper/2602.05895/) (2026)
- [Learning-augmented robotic automation for real-world manufacturing](https://mlsanity.com/paper/2604.22235/) (2026)
- [Bridging the Gap: Enabling Soft Actor Critic for High Performance Legged Locomotion](https://mlsanity.com/paper/2605.24975/) (2026)
- [Expanding Spatial and Temporal Context for Robotic Imitation Learning With Scene Graphs](https://mlsanity.com/paper/2606.01072/) (2026)
- [CoRe-MoE: Contrastive Reweighted Mixture of Experts for Multi-Terrain Humanoid Locomotion with Gait Adaptation](https://mlsanity.com/paper/2606.04718/) (2026)
- [PLUME: Probabilistic Latent Unified World Modeling and Parameter Estimation for Multi-Finger Manipulation](https://mlsanity.com/paper/2606.11396/) (2026)
- [LoComposition: Terrain-Adaptive Energy-Efficient Quadruped Locomotion without Gait Priors](https://mlsanity.com/paper/2606.15896/) (2026)
- [AnnotateAnything: Automatic Annotation of 3D Assets for Robot Manipulation](https://mlsanity.com/paper/2606.17446/) (2026)
- [MagicSim: A Unified Infrastructure for Executable Embodied Interaction](https://mlsanity.com/paper/2606.17511/) (2026)
- [GigaBrain-WBC-0.5: A Behavior World Model for Robust Humanoid Whole-Body Tracking with Environment Interaction](https://mlsanity.com/paper/2608.18234/) (2026)
- [Timestep Weighting: A Hidden Key to Effective ELBO-Based Flow-Matching RL](https://mlsanity.com/paper/2609.32665/) (2026)
- [SkillWeaver: Agentic Exploration over Neural Interaction Skills for Scalable Robot Data Generation](https://mlsanity.com/paper/2609.36171/) (2026)
- [H-JEPA: End-to-End Learning of Hierarchical World Models for Visual Planning](https://mlsanity.com/paper/2610.06805/) (2026)
- [Cooperating with Future Collaborators: Multi-Agent RL under Staggered Participation](https://mlsanity.com/paper/2610.07578/) (2026)
- [Modeling Latent Disturbances for Robust Decision-Making in World Models](https://mlsanity.com/paper/2610.07599/) (2026)
- [Learning Grasp Targeting from Point Clouds for Log Pile Clearing on a Hydraulic Crane](https://mlsanity.com/paper/2610.07613/) (2026)
- [SMART: Zero-Shot Sim-to-Real Articulated Object Manipulation via Large-Scale Synthetic Pretraining](https://mlsanity.com/paper/2610.07652/) (2026)
- [OmniDex: Scaling Dexterous Hand Grasping to Diverse Cluttered Scenes](https://mlsanity.com/paper/2610.11194/) (2026)
- [FAITH: Feasibility-Aware Safety-Filtered RL for High-Dimensional Systems](https://mlsanity.com/paper/2610.12432/) (2026)
- [Dex-One2Many: Learning Dexterous Manipulation from a Single Human Demonstration](https://mlsanity.com/paper/2610.12470/) (2026)

## Abstract (arXiv)

We present Isaac Lab, the natural successor to Isaac Gym, which extends the paradigm of GPU-native robotics simulation into the era of large-scale multi-modal learning. Isaac Lab combines high-fidelity GPU parallel physics, photorealistic rendering, and a modular, composable architecture for designing environments and training robot policies. Beyond physics and rendering, the framework integrates actuator models, multi-frequency sensor simulation, data collection pipelines, and domain randomization tools, unifying best practices for reinforcement and imitation learning at scale within a single extensible platform. We highlight its application to a diverse set of challenges, including whole-body control, cross-embodiment mobility, contact-rich and dexterous manipulation, and the integration of human demonstrations for skill acquisition. Finally, we discuss upcoming integration with the differentiable, GPU-accelerated Newton physics engine, which promises new opportunities for scalable, data-efficient, and gradient-based approaches to robot learning. We believe Isaac Lab's combination of advanced simulation capabilities, rich sensing, and data-center scale execution will help unlock the next generation of breakthroughs in robotics research.