IQ-JEPA: A Joint-Embedding Predictive Architecture with a Hermitian Vision Transformer for Sound Speed and Attenuation Estimation from Ultrasound IQ Data
Not on the map. Why?
Abstract (arXiv)
The speed of sound in tissue is a prerequisite for well-focused imaging and has diagnostic value, but recovering it from raw pulse-echo channel data is fundamentally a nonlinear inverse problem. Learned solvers are fast yet label hungry. Simulated sound-speed labels are expensive, while abundant real channel data is unlabeled. We propose IQ-JEPA to exploit both data types. An encoder is pretrained without labels to predict the latent representation of masked in-phase and quadrature (IQ) regions from visible context, then fine-tuned on simulated maps. Sound speed appears in the IQ signal as a phase difference, invariant to the constant phase offset. The encoder is a Hermitian vision transformer that operates on the complex signal directly. Its attention is equivariant to that phase and its conjugate-product feed-forward is invariant to it, so the encoder reads a quantity analogous to the one classical coherence methods use. On 79,293 Fullwave 2.5 simulations at 2.5 MHz, pretraining on the 63,435 unlabeled acquisitions reaches 15.60 m/s at 10,000 labels. This is a roughly threefold gain in label efficiency over supervised training, growing to over fourfold at 1,000 labels. It is about 2.2x below an InversionNet baseline, and 8.71 m/s at full labels. The gain still grows with more unlabeled pretraining data. Our comparisons point to self-supervision as the dominant factor. The same encoder transfers. Its frozen features expose sound speed and attenuation, and cross-distribution pretraining between layered and abdominal phantoms costs little accuracy. We see this as a first step toward a foundation model for quantitative ultrasound.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 29 of its 82 references that are in the map. First posted on arXiv 24 Jul 2026.
Builds on
29 of 82 references.
References · 29
- 2017Attention Is All You Needcited by 60,361
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
- 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
- 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
- 2023Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecturecited by 1,039
- 2024Revisiting Feature Prediction for Learning Visual Representations from Videocited by 531
- 2017Deep Complex Networkscited by 217
- 2026V-JEPA 2.1: Unlocking Dense Features in Video Self-Supervised Learningcited by 117
- 2018InversionNet: A Real-Time and Accurate Full Waveform Inversion with CNNs and continuous CRFscited by 61
17 more
- 2021OpenFWI: Large-Scale Multi-Structural Benchmark Datasets for Seismic Full Waveform Inversioncited by 44
- 2019Complex Transformer: A Framework for Modeling Complex-Valued Sequencecited by 32
- 2021Unsupervised Learning of Full-Waveform Inversion: Connecting CNN and Partial Differential Equation in a Loopcited by 29
- 2023Building Blocks for a Complex-Valued Transformer Architecturecited by 16
- 2024From Pretraining to Privacy: Federated Ultrasound Foundation Model with Self-Supervised Learningcited by 15
- 2018A Deep Learning Framework for Single-Sided Sound Speed Inversion in Medical Ultrasoundcited by 10
- 2024SeisLM: a Foundation Model for Seismic Waveformscited by 8
- 2018Image Reconstruction via Variational Network for Real-Time Hand-Held Sound-Speed Imagingcited by 6
- 2023Auto-Linear Phenomenon in Subsurface Imagingcited by 6
- 2020Training Variational Networks with Multi-Domain Simulations: Speed-of-Sound Image Reconstructioncited by 5
- 2022Phase Aberration Correction for in vivo Ultrasound Localization Microscopy Using a Spatiotemporal Complex-Valued Neural Networkcited by 4
- 2020Complex Convolutional Neural Networks for Ultrafast Ultrasound Image Reconstruction from In-Phase/Quadrature Signalcited by 3
- 2022Nonlinear Waveform Inversion for Quantitative Ultrasoundcited by 3
- 2024Implicit Neural Representations for Speed-of-Sound Estimation in Ultrasoundcited by 2
- 2025Self-supervised Deep Learning for Denoising in Ultrasound Microvascular Imagingcited by 2
- 2026Efficient Complex-Valued Vision Transformers for MRI Classification Directly from k-Spacecited by 1
- 2026When do complex-valued neural networks help? A study of representation, geometry, and optimizationcited by 1
Built on by
Not cited yet.