mlsanityData to 7 Oct 2026

Translation-based Video-to-Video Synthesis

arXiv 2404.04283 ↗First posted 3 Apr 2024cs.CV cs.LG eess.IVcited by 1

Not on the map. Why?

Abstract (arXiv)

Translation-based Video Synthesis (TVS) has emerged as a vital research area in computer vision, aiming to facilitate the transformation of videos between distinct domains while preserving both temporal continuity and underlying content features. This technique has found wide-ranging applications, encompassing video super-resolution, colorization, segmentation, and more, by extending the capabilities of traditional image-to-image translation to the temporal domain. One of the principal challenges faced in TVS is the inherent risk of introducing flickering artifacts and inconsistencies between frames during the synthesis process. This is particularly challenging due to the necessity of ensuring smooth and coherent transitions between video frames. Efforts to tackle this challenge have induced the creation of diverse strategies and algorithms aimed at mitigating these unwanted consequences. This comprehensive review extensively examines the latest progress in the realm of TVS. It thoroughly investigates emerging methodologies, shedding light on the fundamental concepts and mechanisms utilized for proficient video synthesis. This survey also illuminates their inherent strengths, limitations, appropriate applications, and potential avenues for future development.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 32 of its 59 references that are in the map. First posted on arXiv 3 Apr 2024.

Builds on

32 of 59 references.

References · 32

  1. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  2. 2022Generative Adversarial Networkscited by 5,603
  3. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,845
  4. 2019Semantic Image Synthesis with Spatially-Adaptive Normalizationcited by 1,295
  5. 2016Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,061
  6. 2015FlowNet: Learning Optical Flow with Convolutional Networkscited by 1,054
  7. 2017The 2017 DAVIS Challenge on Video Object Segmentationcited by 942
  8. 2016FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networkscited by 780
  9. 2020A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wildcited by 669
  10. 2017MoCoGAN: Decomposing Motion and Content for Video Generationcited by 523
  11. 2017A Brief Survey of Deep Reinforcement Learningcited by 466
  12. 2015Unsupervised Learning of Video Representations using LSTMscited by 434
20 more
  1. 2018Video-to-Video Synthesiscited by 388
  2. 2018Everybody Dance Nowcited by 313
  3. 2019Few-shot Video-to-Video Synthesiscited by 196
  4. 2017Deep Learning for Precipitation Nowcasting: A Benchmark and A New Modelcited by 180
  5. 2017Playing for Benchmarkscited by 169
  6. 2018Recycle-GAN: Unsupervised Video Retargetingcited by 89
  7. 2019Deep Exemplar-based Video Colorizationcited by 75
  8. 2020World-Consistent Video-to-Video Synthesiscited by 69
  9. 2018Pose Guided Human Video Generationcited by 67
  10. 2019Mocycle-GAN: Unpaired Video-to-Video Translationcited by 37
  11. 2017Radiomics strategies for risk assessment of tumour failure in head-and-neck cancercited by 31
  12. 2021Long-Term Temporally Consistent Unpaired Video Translation from Simulated Surgical 3D Datacited by 21
  13. 2019Preserving Semantic and Temporal Consistency for Unpaired Video-to-Video Translationcited by 20
  14. 2018Controllable Image-to-Video Translation: A Case Study on Facial Expression Generationcited by 12
  15. 2021I2V-GAN: Unpaired Infrared-to-Visible Video Translationcited by 10
  16. 2018Unsupervised Video-to-Video Translationcited by 8
  17. 2020Unsupervised Multimodal Video-to-Video Translation via Self-Supervised Learningcited by 7
  18. 2021Invertible Frowns: Video-to-Video Facial Emotion Translationcited by 6
  19. 2022Learning Temporally and Semantically Consistent Unpaired Video-to-video Translation Through Pseudo-Supervision From Synthetic Optical Flowcited by 5
  20. 2019HyperCon: Image-To-Video Model Transfer for Video-To-Video Translation Taskscited by 2

Built on by

Cited by year

Citing papers per year: 2024: 1.

1 paper.

  1. 2024Video Motion Transfer with Diffusion Transformerscited by 39