OnUVS: Online Feature Decoupling Framework for High-Fidelity Ultrasound Video Synthesis
Not on the map. Why?
Abstract (arXiv)
Ultrasound (US) imaging is indispensable in clinical practice. To diagnose certain diseases, sonographers must observe corresponding dynamic anatomic structures to gather comprehensive information. However, the limited availability of specific US video cases causes teaching difficulties in identifying corresponding diseases, which potentially impacts the detection rate of such cases. The synthesis of US videos may represent a promising solution to this issue. Nevertheless, it is challenging to accurately animate the intricate motion of dynamic anatomic structures while preserving image fidelity. To address this, we present a novel online feature-decoupling framework called OnUVS for high-fidelity US video synthesis. Our highlights can be summarized by four aspects. First, we introduced anatomic information into keypoint learning through a weakly-supervised training strategy, resulting in improved preservation of anatomical integrity and motion while minimizing the labeling burden. Second, to better preserve the integrity and textural information of US images, we implemented a dual-decoder that decouples the content and textural features in the generator. Third, we adopted a multiple-feature discriminator to extract a comprehensive range of visual cues, thereby enhancing the sharpness and fine details of the generated videos. Fourth, we constrained the motion trajectories of keypoints during online learning to enhance the fluidity of generated videos. Our validation and user studies on in-house echocardiographic and pelvic floor US videos showed that OnUVS synthesizes US videos with high fidelity.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 27 of its 47 references that are in the map. First posted on arXiv 16 Aug 2023.
Builds on
27 of 47 references.
References · 27
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
- 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
- 2022Generative Adversarial Networkscited by 5,627
- 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,527
- 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,270
- 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,694
- 2018SlowFast Networks for Video Recognitioncited by 1,867
- 2016Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learningcited by 1,512
- 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,234
- 2017High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANscited by 1,179
15 more
- 2017A Closer Look at Spatiotemporal Convolutions for Action Recognitioncited by 1,143
- 2022Diffusion Models in Vision: A Surveycited by 1,015
- 2016Least Squares Generative Adversarial Networkscited by 1,001
- 2020First Order Motion Model for Image Animationcited by 704
- 2021Do Vision Transformers See Like Convolutional Neural Networks?cited by 675
- 2021Motion Representations for Articulated Animationcited by 255
- 2022Thin-Plate Spline Motion Model for Image Animationcited by 210
- 2023Dreamix: Video Diffusion Models are General Video Editorscited by 191
- 2022CrossMoDA 2021 challenge: Benchmark of Cross-Modality Domain Adaptation techniques for Vestibular Schwannoma and Cochlea Segmentationcited by 58
- 2022Sketch guided and progressive growing GAN for realistic and editable ultrasound image synthesiscited by 42
- 2022Structure-Aware Motion Transfer with Deformable Anchor Modelcited by 34
- 2022Weakly-supervised High-fidelity Ultrasound Video Synthesis with Feature Decouplingcited by 6
- 2021Realistic Ultrasound Image Synthesis for Improved Classification of Liver Diseasecited by 5
- 2022Explainable Ensemble Machine Learning for Breast Cancer Diagnosis based on Ultrasound Image Texture Featurescited by 5
- 2023Inflated 3D Convolution-Transformer for Weakly-supervised Carotid Stenosis Grading with Ultrasound Videoscited by 3
Built on by
Citing papers per year: 2024: 1.
1 paper.