mlsanityData to 14 Oct 2026

LatentQuant: Preserving the Policy-Facing Latent Contract under NVFP4 VAE Quantization

arXiv 2610.03959 ↗First posted 2 Oct 2026cs.AIcited by 0

On the map: World models → Latent predictive world models also fits: Embodied and robot world models

Abstract (arXiv)

Recent world action models (WAMs) reuse pretrained video VAEs whose encoder latents directly condition downstream action policies. Quantization must therefore preserve not only reconstruction fidelity but also the policy-facing latent contract expected by the frozen policy. Direct NVFP4 leaves W4A4 quantization error uncompensated, whereas joint quantization-aware training (QAT) can recover reconstruction by moving this representation. On Wan2.1, joint QAT nearly matches FP32 VBench-7 (0.7403 versus 0.7409), yet LIBERO success collapses from 95.5% to 10.5%. Controlled decoder-only experiments show that activation quantize-dequantize operations alter the reconstruction signal and decoder Jacobian, redirecting the gradient returned to the encoder and inducing persistent latent drift. Based on this mechanism, we introduce LatentQuant, a two-stage NVFP4 QAT framework that first aligns the quantized encoder with its high-precision counterpart, then freezes it while adapting the decoder. Across Wan2.1 and Wan2.2, LatentQuant preserves near-baseline control and high reconstruction quality, achieving 95.75% success on LIBERO and 68.8% on RoboTwin. On NVIDIA B300 GPUs, NVFP4 execution achieves 1.17x-1.26x end-to-end VAE speedups over BF16 cuDNN.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Latent predictive world models, one of 11 directions in World models. Builds on 23 of its 27 references that are in the map. First posted on arXiv 2 Oct 2026.

Builds on

23 of 27 references.

Closest ancestors · 1

Same direction, newest first

  1. 2026An Empirical Study of World Model Quantization2 Feb

Other references · 22

  1. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,822
  2. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,593
  3. 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,410
  4. 2025RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulationcited by 518
  5. 2025Reconstruction vs. Generation: Taming Optimization Dilemma in Latent Diffusion Modelscited by 393
  6. 2024GR-2: A Generative Video-Language-Action Model with Web-Scale Knowledge for Robot Manipulationcited by 296
  7. 2024Deep Compression Autoencoder for Efficient High-Resolution Diffusion Modelscited by 217
  8. 2024SVDQuant: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Modelscited by 170
  9. 2024ViDiT-Q: Efficient and Accurate Quantization of Diffusion Transformers for Image and Video Generationcited by 115
  10. 2025EQ-VAE: Equivariance Regularized Latent Space for Improved Generative Image Modelingcited by 89
  11. 2025Improving the Diffusability of Autoencoderscited by 80
  12. 2024CV-VAE: A Compatible Video VAE for Latent Generative Video Modelscited by 70
10 more
  1. 2025DC-AE 1.5: Accelerating Diffusion Model Convergence with Structured Latent Spacecited by 47
  2. 2026QuantVLA: Scale-Calibrated Post-Training Quantization for Vision-Language-Action Modelscited by 30
  3. 2025QVGen: Pushing the Limit of Quantized Video Generative Modelscited by 22
  4. 2026Quantization-Aware Distillation for NVFP4 Inference Accuracy Recoverycited by 22
  5. 2026WALL-WM: Carving World Action Modeling at the Event Jointscited by 17
  6. 2025H3AE: High Compression, High Speed, and High Quality AutoEncoder for Video Diffusion Modelscited by 13
  7. 2024PassionSR: Post-Training Quantization with Adaptive Scale in One-Step Diffusion based Image Super-Resolutioncited by 9
  8. 2026Flash-VAED: Plug-and-Play VAE Decoders for Efficient Video Generationcited by 7
  9. 2026DA-VAE: Plug-in Latent Compression for Diffusion via Detail Alignmentcited by 6
  10. 2026Rethinking Shrinkage Bias in LLM FP4 Pretraining: Geometric Origin, Systemic Impact, and UFP4 Recipecited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Q-WAM: 4-Bit Quantization of World Action Models with Action-Subspace Protection
  2. 2026LAFP: Preserving Latent Action Structure in Latent Policy Learning via Flow Matching
  3. 2026VLAQuantBench: Closed-Loop Evaluation of Post-Training Quantization for Vision-Language-Action Models
  4. 2026ALAM: Algebraically Consistent Latent Action Model for Vision-Language-Action Models
  5. 2026SteerQuant: Steering Quantization Error with Action-Guided Scaling in World-Action Models
  6. 2026ActQuant: Sub-4-bit Action-Guided Quantization for Vision-Language-Action Models
  7. 2026Recovering Low-Bit Failures in Vision-Language-Action Policies via Quantization-Conditioned Reinforcement Learning
  8. 2026Timestep-Aware SVDQuant-GPTQ for W4A4 Quantization of Wan2.2-I2V