# Soft-Sensing ConFormer: A Curriculum Learning-based Convolutional Transformer

arXiv: https://arxiv.org/abs/2111.06981 · HTML: https://mlsanity.com/paper/2111.06981/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 31 references that are in the map.
- First posted on arXiv 12 Nov 2021.

## Builds on (most cited first, 12 of 12)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [Multi-Scale Context Aggregation by Dilated Convolutions](https://mlsanity.com/paper/1511.07122/) (2015)
- [LSTM Fully Convolutional Networks for Time Series Classification](https://mlsanity.com/paper/1709.05206/) (2017)
- [Lite Transformer with Long-Short Range Attention](https://mlsanity.com/paper/2004.11886/) (2020)
- [Multivariate LSTM-FCNs for Time Series Classification](https://mlsanity.com/paper/1801.04503/) (2018)
- [ContextNet: Improving Convolutional Neural Networks for Automatic Speech Recognition with Global Context](https://mlsanity.com/paper/2005.03191/) (2020)
- [ConvTransformer: A Convolutional Transformer Network for Video Frame Synthesis](https://mlsanity.com/paper/2011.10185/) (2020)
- [Auto-encoder based Model for High-dimensional Imbalanced Industrial Data](https://mlsanity.com/paper/2108.02083/) (2021)
- [Soft Sensing Transformer: Hundreds of Sensors are Worth a Single Word](https://mlsanity.com/paper/2111.05973/) (2021)

## Built on by (most cited first, 4 of 4)

- [Soft Sensing Regression Model: from Sensor to Wafer Metrology Forecasting](https://mlsanity.com/paper/2301.08974/) (2023)
- [Wafer Defect Root Cause Analysis with Partial Trajectory Regression](https://mlsanity.com/paper/2507.20357/) (2025)
- [Sequence-Aware Inline Measurement Attribution for Good-Bad Wafer Diagnosis](https://mlsanity.com/paper/2507.20364/) (2025)
- [Cross-Process Defect Attribution using Potential Loss Analysis](https://mlsanity.com/paper/2508.00895/) (2025)

## Abstract (arXiv)

Over the last few decades, modern industrial processes have investigated several cost-effective methodologies to improve the productivity and yield of semiconductor manufacturing. While playing an essential role in facilitating real-time monitoring and control, the data-driven soft-sensors in industries have provided a competitive edge when augmented with deep learning approaches for wafer fault-diagnostics. Despite the success of deep learning methods across various domains, they tend to suffer from bad performance on multi-variate soft-sensing data domains. To mitigate this, we propose a soft-sensing ConFormer (CONvolutional transFORMER) for wafer fault-diagnostic classification task which primarily consists of multi-head convolution modules that reap the benefits of fast and light-weight operations of convolutions, and also the ability to learn the robust representations through multi-head design alike transformers. Another key issue is that traditional learning paradigms tend to suffer from low performance on noisy and highly-imbalanced soft-sensing data. To address this, we augment our soft-sensing ConFormer model with a curriculum learning-based loss function, which effectively learns easy samples in the early phase of training and difficult ones later. To further demonstrate the utility of our proposed architecture, we performed extensive experiments on various toolsets of Seagate Technology's wafer manufacturing process which are shared openly along with this work. To the best of our knowledge, this is the first time that curriculum learning-based soft-sensing ConFormer architecture has been proposed for soft-sensing data and our results show strong promise for future use in soft-sensing research domain.