# Represent, Then Generate: Multimodal-Conditioned Time-Series Generation under Irregular Missingness

arXiv: https://arxiv.org/abs/2608.12592 · HTML: https://mlsanity.com/paper/2608.12592/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 38 references that are in the map.
- First posted on arXiv 12 Aug 2026.

## Builds on (most cited first, 20 of 20)

- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow](https://mlsanity.com/paper/2209.03003/) (2022)
- [Multimodal Machine Learning: A Survey and Taxonomy](https://mlsanity.com/paper/1705.09406/) (2017)
- [CSDI: Conditional Score-based Diffusion Models for Probabilistic Time Series Imputation](https://mlsanity.com/paper/2107.03502/) (2021)
- [Recurrent Neural Networks for Multivariate Time Series with Missing Values](https://mlsanity.com/paper/1606.01865/) (2016)
- [Multitask learning and benchmarking with clinical time series data](https://mlsanity.com/paper/1703.07771/) (2017)
- [Diffusion-TS: Interpretable Diffusion for General Time Series Generation](https://mlsanity.com/paper/2403.01742/) (2024)
- [Vector Quantized Time Series Generation with a Bidirectional Prior Model](https://mlsanity.com/paper/2303.04743/) (2023)
- [Utilizing Image Transforms and Diffusion Models for Generative Modeling of Short and Long Time Series](https://mlsanity.com/paper/2410.19538/) (2024)
- [Time Weaver: A Conditional Time Series Generation Model](https://mlsanity.com/paper/2403.02682/) (2024)
- [CardioGAN: Attentive Generative Adversarial Network with Dual Discriminators for Synthesis of ECG from PPG](https://mlsanity.com/paper/2010.00104/) (2020)
- [TTS-CGAN: A Transformer Time-Series Conditional GAN for Biosignal Data Augmentation](https://mlsanity.com/paper/2206.13676/) (2022)
- [T2S: High-resolution Time Series Generation with Text-to-Series Diffusion Models](https://mlsanity.com/paper/2505.02417/) (2025)
- [BRIDGE: Bootstrapping Text to Control Time-Series Generation via Multi-Agent Iterative Optimization and Diffusion Modeling](https://mlsanity.com/paper/2503.02445/) (2025)
- [DiffuSETS: 12-lead ECG Generation Conditioned on Clinical Text Reports and Patient-Specific Information](https://mlsanity.com/paper/2501.05932/) (2025)
- [Predicting Intensive Care Unit Length of Stay and Mortality Using Patient Vital Signs: Machine Learning Model Development and Validation](https://mlsanity.com/paper/2105.04414/) (2021)
- [WaveStitch: Flexible and Fast Conditional Time Series Generation with Diffusion Models](https://mlsanity.com/paper/2503.06231/) (2025)
- [ConTSG-Bench: A Unified Benchmark for Conditional Time Series Generation](https://mlsanity.com/paper/2603.04767/) (2026)

## Abstract (arXiv)

Continuous physiological time series underpin modern clinical monitoring, yet many of the most informative signals are invasive, expensive, or simply unavailable for a given patient. Conditional generation offers a remedy: an absent signal can be synthesized from co-recorded signals and routine clinical variables. Existing generators, however, are built around a single conditioning modality and degrade when forced to handle the heterogeneous, irregularly missing mix of time-variant signals and static covariates seen in practice. We propose ReCoGen (Represent Conditions, then Generate), a two-stage framework that decouples multimodal condition representation from target generation. Stage I trains one masked autoencoder per modality, distilling each time-variant condition into a compact and missingness-tolerant token sequence. Stage II trains a flow-matching generator that fuses these tokens with static conditions to synthesize the target signal. Across three physiological benchmarks, including continuous glucose monitoring on AI-READI and arterial blood pressure generation on MIMIC-III and MIMIC-IV, ReCoGen attains the best downstream utility on all sixteen (dataset, task, metric) settings, surpassing six representative conditional generators; on thirteen of them its utility also reaches or exceeds the utility measured on the real signal, a reference we read as an approximate anchor rather than a ceiling. Ablations trace the gains to the conditioning path: learnable cross-attention over the frozen per-modality encoders, and a dual token-plus-AdaLN route for the static conditions. ReCoGen thus turns routinely collected signals into informative surrogates for invasive or unavailable ones, a step toward less invasive, lower-cost continuous clinical monitoring.