# SudokuSens: Enhancing Deep Learning Robustness for IoT Sensing Applications using a Generative Approach

arXiv: https://arxiv.org/abs/2402.02275 · HTML: https://mlsanity.com/paper/2402.02275/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 45 references that are in the map.
- First posted on arXiv 3 Feb 2024.

## Builds on (most cited first, 25 of 25)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [A Survey on Contrastive Self-supervised Learning](https://mlsanity.com/paper/2011.00362/) (2020)
- [TS2Vec: Towards Universal Representation of Time Series](https://mlsanity.com/paper/2106.10466/) (2021)
- [Real-valued (Medical) Time Series Generation with Recurrent Conditional GANs](https://mlsanity.com/paper/1706.02633/) (2017)
- [Unsupervised Feature Extraction by Time-Contrastive Learning and Nonlinear ICA](https://mlsanity.com/paper/1605.06336/) (2016)
- [Self-Supervised Contrastive Pre-Training For Time Series via Time-Frequency Consistency](https://mlsanity.com/paper/2206.08496/) (2022)
- [Time Series Data Augmentation for Deep Learning: A Survey](https://mlsanity.com/paper/2002.12478/) (2020)
- [Unsupervised Representation Learning for Time Series with Temporal Neighborhood Coding](https://mlsanity.com/paper/2106.00750/) (2021)
- [Towards Fairer Datasets: Filtering and Balancing the Distribution of the People Subtree in the ImageNet Hierarchy](https://mlsanity.com/paper/1912.07726/) (2019)
- [TCLR: Temporal Contrastive Learning for Video Representation](https://mlsanity.com/paper/2101.07974/) (2021)
- [Data Augmentation of Wearable Sensor Data for Parkinson's Disease Monitoring using Convolutional Neural Networks](https://mlsanity.com/paper/1706.00527/) (2017)
- [Multi-Scale Convolutional Neural Networks for Time Series Classification](https://mlsanity.com/paper/1603.06995/) (2016)
- [Attribute2Image: Conditional Image Generation from Visual Attributes](https://mlsanity.com/paper/1512.00570/) (2015)
- [Learning to Compose Domain-Specific Transformations for Data Augmentation](https://mlsanity.com/paper/1709.01643/) (2017)
- [DeepSense: A Unified Deep Learning Framework for Time-Series Mobile Sensing Data Processing](https://mlsanity.com/paper/1611.01942/) (2016)
- [Exploring Contrastive Learning in Human Activity Recognition for Healthcare](https://mlsanity.com/paper/2011.11542/) (2020)
- [Assessing the State of Self-Supervised Human Activity Recognition using Wearables](https://mlsanity.com/paper/2202.12938/) (2022)
- [Time Series Anomaly Detection Using Convolutional Neural Networks and Transfer Learning](https://mlsanity.com/paper/1905.13628/) (2019)
- [STFNets: Learning Sensing Signals from the Time-Frequency Perspective with Short-Time Fourier Neural Networks](https://mlsanity.com/paper/1902.07849/) (2019)
- [IMUTube: Automatic Extraction of Virtual on-body Accelerometry from Video for Human Activity Recognition](https://mlsanity.com/paper/2006.05675/) (2020)
- [COCOA: Cross Modality Contrastive Learning for Sensor Data](https://mlsanity.com/paper/2208.00467/) (2022)
- [Learning to Sample and Aggregate: Few-shot Reasoning over Temporal Knowledge Graphs](https://mlsanity.com/paper/2210.08654/) (2022)
- [DyDiff-VAE: A Dynamic Variational Framework for Information Diffusion Prediction](https://mlsanity.com/paper/2106.03251/) (2021)
- [Unsupervised Belief Representation Learning with Information-Theoretic Variational Graph Auto-Encoders](https://mlsanity.com/paper/2110.00210/) (2021)

## Built on by (most cited first, 2 of 2)

- [Hawk: An Efficient NALM System for Accurate Low-Power Appliance Recognition](https://mlsanity.com/paper/2410.16293/) (2024)
- [Responsive DNN Adaptation for Video Analytics against Environment Shift via Hierarchical Mobile-Cloud Collaborations](https://mlsanity.com/paper/2505.00745/) (2025)

## Abstract (arXiv)

This paper introduces SudokuSens, a generative framework for automated generation of training data in machine-learning-based Internet-of-Things (IoT) applications, such that the generated synthetic data mimic experimental configurations not encountered during actual sensor data collection. The framework improves the robustness of resulting deep learning models, and is intended for IoT applications where data collection is expensive. The work is motivated by the fact that IoT time-series data entangle the signatures of observed objects with the confounding intrinsic properties of the surrounding environment and the dynamic environmental disturbances experienced. To incorporate sufficient diversity into the IoT training data, one therefore needs to consider a combinatorial explosion of training cases that are multiplicative in the number of objects considered and the possible environmental conditions in which such objects may be encountered. Our framework substantially reduces these multiplicative training needs. To decouple object signatures from environmental conditions, we employ a Conditional Variational Autoencoder (CVAE) that allows us to reduce data collection needs from multiplicative to (nearly) linear, while synthetically generating (data for) the missing conditions. To obtain robustness with respect to dynamic disturbances, a session-aware temporal contrastive learning approach is taken. Integrating the aforementioned two approaches, SudokuSens significantly improves the robustness of deep learning for IoT applications. We explore the degree to which SudokuSens benefits downstream inference tasks in different data sets and discuss conditions under which the approach is particularly effective.