Codebook Capacity Governs Perceptual Quality Across Resolutions in Hierarchical Discrete Video Compression

Learned video codecs based on continuous latent representations typically require resolution-specific retraining or rate-distortion (RD) recalibration when scaling to new spatial resolutions, because entropy models and Lagrangian weights are tightly coupled to the operating point. We investigate whether hierarchical discrete latent codecs exhibit the same sensitivity. Using a controlled empirical study of MS-VQ-VAE video compression across codebook sizes K ∈ 128,256,512,1024 and resolutions 64×64, 128×128, and 256×256 on UCF101, we show that perceptual quality (LPIPS) depends strongly on codebook capacity but only negligibly on spatial resolution. Fitting a log-linear model Q(K,r) = αlog2 K + βlog2 r + γ to all 12 operating points yields α=-0.0094 (t=-6.6, p<0.001) and β=-0.0009 (t=-0.43, p=0.68, not significant), with R2=0.82. Codebook capacity is therefore roughly 10× more influential than spatial resolution per log-unit increase. In parallel, bottom-level entropy efficiency η=H(z)/log2 K remains stable or improves with resolution (84-87% at 64×64; 92-94% at 256×256), confirming that larger spatial grids are utilized more efficiently rather than less. Across all resolutions and codebook sizes, our models outperform H.264 on LPIPS at matched or lower bitrate, with gains of 25-52% at 128×128 and 21-37% over H.265 at 256×256. These findings suggest that codebook size K, not spatial resolution, is the dominant design variable governing perceptual compression quality in hierarchical discrete video codecs -- a property that may simplify multi-resolution deployment and inform the design of scalable discrete tokenizers for generative video models.