# Cross-Modal Virtual Sensing for Combustion Instability Monitoring

arXiv: https://arxiv.org/abs/2110.01659 · HTML: https://mlsanity.com/paper/2110.01659/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 43 references that are in the map.
- First posted on arXiv 4 Oct 2021.

## Builds on (most cited first, 10 of 10)

- [Conditional Generative Adversarial Nets](https://mlsanity.com/paper/1411.1784/) (2014)
- [Generative Adversarial Text to Image Synthesis](https://mlsanity.com/paper/1605.05396/) (2016)
- [AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networks](https://mlsanity.com/paper/1711.10485/) (2017)
- [Object-driven Text-to-Image Synthesis via Adversarial Training](https://mlsanity.com/paper/1902.10740/) (2019)
- [Deep Cross-Modal Audio-Visual Generation](https://mlsanity.com/paper/1704.08292/) (2017)
- [Speech2Face: Learning the Face Behind a Voice](https://mlsanity.com/paper/1905.09773/) (2019)
- [Spatiotemporal Attention for Multivariate Time Series Prediction and Interpretation](https://mlsanity.com/paper/2008.04882/) (2020)
- [Wav2Pix: Speech-conditioned Face Generation using Generative Adversarial Networks](https://mlsanity.com/paper/1903.10195/) (2019)
- [Learning to Have an Ear for Face Super-Resolution](https://mlsanity.com/paper/1909.12780/) (2019)
- [3D Convolutional Selective Autoencoder For Instability Detection in Combustion Systems](https://mlsanity.com/paper/2101.01877/) (2021)

## Built on by (most cited first, 1 of 1)

- [A Deep Learning Approach to Detect Lean Blowout in Combustion Systems](https://mlsanity.com/paper/2208.01871/) (2022)

## Abstract (arXiv)

In many cyber-physical systems, imaging can be an important but expensive or 'difficult to deploy' sensing modality. One such example is detecting combustion instability using flame images, where deep learning frameworks have demonstrated state-of-the-art performance. The proposed frameworks are also shown to be quite trustworthy such that domain experts can have sufficient confidence to use these models in real systems to prevent unwanted incidents. However, flame imaging is not a common sensing modality in engine combustors today. Therefore, the current roadblock exists on the hardware side regarding the acquisition and processing of high-volume flame images. On the other hand, the acoustic pressure time series is a more feasible modality for data collection in real combustors. To utilize acoustic time series as a sensing modality, we propose a novel cross-modal encoder-decoder architecture that can reconstruct cross-modal visual features from acoustic pressure time series in combustion systems. With the "distillation" of cross-modal features, the results demonstrate that the detection accuracy can be enhanced using the virtual visual sensing modality. By providing the benefit of cross-modal reconstruction, our framework can prove to be useful in different domains well beyond the power generation and transportation industries.