mlsanityData to 7 Oct 2026

VCT: A Video Compression Transformer

arXiv 2206.07307 ↗First posted 15 Jun 2022cs.CV cs.LG eess.IVcited by 74

Not on the map. Why?

Abstract (arXiv)

We show how transformers can be used to vastly simplify neural video compression. Previous methods have been relying on an increasing number of architectural biases and priors, including motion prediction and warping operations, resulting in complex models. Instead, we independently map input frames to representations and use a transformer to model their dependencies, letting it predict the distribution of future representations given the past. The resulting video compression transformer outperforms previous methods on standard video compression data sets. Experiments on synthetic data show that our model learns to handle complex motion patterns such as panning, blurring and fading purely from data. Our approach is easy to implement, and we release code to facilitate future research.

Key facts. Cited by 74 papers in the map (data through 7 Oct 2026). Builds on 34 of its 60 references that are in the map. First posted on arXiv 15 Jun 2022.

Builds on

34 of 60 references.

References · 34

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  6. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  7. 2021ViViT: A Video Vision Transformercited by 1,610
  8. 2021Is Space-Time Attention All You Need for Video Understanding?cited by 1,609
  9. 2020Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformerscited by 1,368
  10. 2018Variational image compression with a scale hyperpriorcited by 910
  11. 2021Multiscale Vision Transformerscited by 896
  12. 2019VideoBERT: A Joint Model for Video and Language Representation Learningcited by 702
22 more
  1. 2018Joint Autoregressive and Hierarchical Priors for Learned Image Compressioncited by 642
  2. 2019Making Convolutional Networks Shift-Invariant Againcited by 390
  3. 2020MaX-DeepLab: End-to-End Panoptic Segmentation with Mask Transformerscited by 355
  4. 2018DVC: An End-to-end Deep Video Compression Frameworkcited by 324
  5. 2022ELIC: Efficient Learned Image Compression with Unevenly Grouped Space-Channel Contextual Adaptive Codingcited by 296
  6. 2017Lossy Image Compression with Compressive Autoencoderscited by 289
  7. 2021Video Transformer Networkcited by 271
  8. 2020Channel-wise Autoregressive Entropy Models for Learned Image Compressioncited by 255
  9. 2021Deep Contextual Video Compressioncited by 216
  10. 2021FVC: A New Framework towards Deep Video Compression in Feature Spacecited by 149
  11. 2018Video Compression through Image Interpolationcited by 143
  12. 2020Nonlinear Transform Codingcited by 132
  13. 2022Entroformer: A Transformer-based Entropy Model for Learned Image Compressioncited by 102
  14. 2019Video Compression With Rate-Distortion Autoencoderscited by 97
  15. 2022An Introduction to Neural Data Compressioncited by 78
  16. 2021ELF-VC: Efficient Learned Flexible-Rate Video Codingcited by 68
  17. 2020Learning for Video Compression with Recurrent Auto-Encoder and Recurrent Probability Modelcited by 67
  18. 2020Conditional Entropy Coding for Efficient Video Compressioncited by 42
  19. 2021Perceptual Learned Video Compression with Recurrent Conditional GANcited by 38
  20. 2021Neural Video Compression using GANs for Detail Synthesis and Propagationcited by 35
  21. 2020Feedback Recurrent Autoencoder for Video Compressioncited by 30
  22. 2020Hierarchical Autoregressive Modeling for Neural Video Compressioncited by 28

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 20; 2024: 22; 2025: 22; 2026: 7.

74 papers.

Sort:

By area:

  1. 2023Neural Video Compression with Diverse Contextscited by 155
  2. 2024Neural Video Compression with Feature Modulationcited by 115
  3. 2024Image and Video Tokenization with Binary Spherical Quantizationcited by 110
  4. 2025Improving the Diffusability of Autoencoderscited by 74
  5. 2025Towards Practical Real-Time Neural Video Compressioncited by 72
  6. 2023HiNeRV: Video Compression with Hierarchical Encoding-based Neural Representationcited by 70
  7. 2023Frequency-Aware Transformer for Learned Image Compressioncited by 56
  8. 2023C3: High-performance and low-complexity neural compression from a single image or videocited by 54
  9. 2024ElasticTok: Adaptive Tokenization for Image and Videocited by 44
  10. 2024A Survey on Generative AI and LLM for Video Generation, Understanding, and Streamingcited by 37
  11. 2024NVRC: Neural Video Representation Compressioncited by 29
  12. 2025GIFStream: 4D Gaussian-based Immersive Video with Feature Streamcited by 23
38 more
  1. 2023Computationally-Efficient Neural Image Compression with Shallow Decoderscited by 22
  2. 2024Spatial Decomposition and Temporal Fusion based Inter Prediction for Learned Video Compressioncited by 21
  3. 2024Bi-Directional Deep Contextual Video Compressioncited by 20
  4. 2022B-CANF: Adaptive B-frame Coding with Conditional Augmented Normalizing Flowscited by 18
  5. 2023M2T: Masking Transformers Twice for Faster Decodingcited by 14
  6. 2023VNVC: A Versatile Neural Video Coding Framework for Efficient Human-Machine Visioncited by 13
  7. 2023IBVC: Interpolation-driven B-frame Video Compressioncited by 12
  8. 2024Group-aware Parameter-efficient Updating for Content-Adaptive Neural Video Compressioncited by 12
  9. 2023MobileNVC: Real-time 1080p Neural Video Compression on a Mobile Devicecited by 11
  10. 2024SMC++: Masked Learning of Unsupervised Video Semantic Compressioncited by 11
  11. 2025Lossy Neural Compression for Geospatial Analytics: A Reviewcited by 10
  12. 2026OneVision-Encoder: Codec-Aligned Sparsity as a Foundational Principle for Multimodal Intelligencecited by 10
  13. 2023MMVC: Learned Multi-Mode Video Compression with Block-based Prediction Mode Selection and Density-Adaptive Entropy Codingcited by 9
  14. 2023Predictive Coding For Animation-Based Video Compressioncited by 9
  15. 2023Towards Real-Time Neural Video Codec for Cross-Platform Application Using Calibration Informationcited by 9
  16. 2025GIViC: Generative Implicit Video Compressioncited by 9
  17. 2025Augmented Deep Contexts for Spatially Embedded Video Codingcited by 8
  18. 2025BiECVC: Gated Diversification of Bidirectional Contexts for Learned Video Compressioncited by 8
  19. 2025Generative Latent Video Compressioncited by 8
  20. 2023Scene Matters: Model-based Deep Video Compressioncited by 6
  21. 2024Benchmarking Conventional and Learned Video Codecs with a Low-Delay Configurationcited by 6
  22. 2025Content-Aware Mamba for Learned Image Compressioncited by 5
  23. 2025Neural B-frame Video Compression with Bi-directional Reference Harmonizationcited by 5
  24. 2023Neural Image Compression Using Masked Sparse Visual Representationcited by 4
  25. 2023Accelerating Learnt Video Codecs with Gradient Decay and Layer-wise Distillationcited by 4
  26. 2026Diffusion-aided Extreme Video Compression with Lightweight Semantics Guidancecited by 4
  27. 2022The Need for Medically Aware Video Compression in Gastroenterologycited by 3
  28. 2024NERV++: An Enhanced Implicit Neural Video Representationcited by 3
  29. 2024Prediction and Reference Quality Adaptation for Learned Video Compressioncited by 3
  30. 2024NVC-1B: A Large Neural Video Coding Modelcited by 3
  31. 2025Token Compression Meets Compact Vision Transformers: A Survey and Comparative Evaluation for Edge AIcited by 3
  32. 2025Context Guided Transformer Entropy Modeling for Video Compressioncited by 3
  33. 2026Ultra-Fast Neural Video Compressioncited by 3
  34. 2026Enhanced Neural Video Representation Compression Across Extreme Complexity and Quality Scalescited by 3
  35. 2023Neural Video Compression with Temporal Layer-Adaptive Hierarchical B-frame Codingcited by 2
  36. 2024Survey on Visual Signal Coding and Processing with Generative Models: Technologies, Standards and Optimizationcited by 2
  37. 2024Standard compliant video coding using low complexity, switchable neural wrapperscited by 2
  38. 2024High-Efficiency Neural Video Compression via Hierarchical Predictive Learningcited by 2

50 most cited of 74 All 74 (JSON)