# Transfer: Cross Modality Knowledge Transfer using Adversarial Networks -- A Study on Gesture Recognition

arXiv: https://arxiv.org/abs/2306.15114 · HTML: https://mlsanity.com/paper/2306.15114/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 86 references that are in the map.
- First posted on arXiv 26 Jun 2023.

## Builds on (most cited first, 25 of 25)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Transferable Features with Deep Adaptation Networks](https://mlsanity.com/paper/1502.02791/) (2015)
- [Adversarial Discriminative Domain Adaptation](https://mlsanity.com/paper/1702.05464/) (2017)
- [Deep CORAL: Correlation Alignment for Deep Domain Adaptation](https://mlsanity.com/paper/1607.01719/) (2016)
- [Deep Domain Confusion: Maximizing for Domain Invariance](https://mlsanity.com/paper/1412.3474/) (2014)
- [Return of Frustratingly Easy Domain Adaptation](https://mlsanity.com/paper/1511.05547/) (2015)
- [Generate To Adapt: Aligning Domains using Generative Adversarial Networks](https://mlsanity.com/paper/1704.01705/) (2017)
- [Towards Accurate Multi-person Pose Estimation in the Wild](https://mlsanity.com/paper/1701.01779/) (2017)
- [Multi-Source Deep Domain Adaptation with Weak Supervision for Time-Series Sensor Data](https://mlsanity.com/paper/2005.10996/) (2020)
- [Domain Adaptation for Time Series Forecasting via Attention Sharing](https://mlsanity.com/paper/2102.06828/) (2021)
- [Transferring Cross-domain Knowledge for Video Sign Language Recognition](https://mlsanity.com/paper/2003.03703/) (2020)
- [Unsupervised Domain Adaptation via Structured Prediction Based Selective Pseudo-Labeling](https://mlsanity.com/paper/1911.07982/) (2019)
- [Towards Domain-Independent and Real-Time Gesture Recognition Using mmWave Signal](https://mlsanity.com/paper/2111.06195/) (2021)
- [Domain Adaptation for sEMG-based Gesture Recognition with Recurrent Neural Networks](https://mlsanity.com/paper/1901.06958/) (2019)
- [Cycle-consistent Conditional Adversarial Transfer Networks](https://mlsanity.com/paper/1909.07618/) (2019)
- [Zero-Shot Sign Language Recognition: Can Textual Data Uncover Sign Languages?](https://mlsanity.com/paper/1907.10292/) (2019)
- [Unifying Unsupervised Domain Adaptation and Zero-Shot Visual Recognition](https://mlsanity.com/paper/1903.10601/) (2019)
- [Unsupervised Domain Adaptation for RF-based Gesture Recognition](https://mlsanity.com/paper/2111.10602/) (2021)
- [EdGCon: Auto-assigner of Iconicity Ratings Grounded by Lexical Properties to Aid in Generation of Technical Gestures](https://mlsanity.com/paper/2306.01944/) (2023)
- [iWash: A Smartwatch Handwashing Quality Assessment and Reminder System with Real-time Feedback in the Context of Infectious Disease](https://mlsanity.com/paper/2009.10317/) (2020)
- [Improved Static Hand Gesture Classification on Deep Convolutional Neural Networks using Novel Sterile Training Technique](https://mlsanity.com/paper/2305.02039/) (2023)
- [Merging Deep Learning with Expert Knowledge for Seizure Onset Zone localization from rs-fMRI in Pediatric Pharmaco Resistant Epilepsy](https://mlsanity.com/paper/2306.05572/) (2023)
- [Few-Shot Domain Adaptation For End-to-End Communication](https://mlsanity.com/paper/2108.00874/) (2021)
- [FaceOff: Detecting Face Touching with a Wrist-Worn Accelerometer](https://mlsanity.com/paper/2008.01769/) (2020)
- [Domain Adaptive Robotic Gesture Recognition with Unsupervised Kinematic-Visual Data Alignment](https://mlsanity.com/paper/2103.04075/) (2021)

## Built on by (most cited first, 1 of 1)

- [STORM: Strategic Orchestration of Modalities for Rare Event Classification](https://mlsanity.com/paper/2412.02805/) (2024)

## Abstract (arXiv)

Knowledge transfer across sensing technology is a novel concept that has been recently explored in many application domains, including gesture-based human computer interaction. The main aim is to gather semantic or data driven information from a source technology to classify / recognize instances of unseen classes in the target technology. The primary challenge is the significant difference in dimensionality and distribution of feature sets between the source and the target technologies. In this paper, we propose TRANSFER, a generic framework for knowledge transfer between a source and a target technology. TRANSFER uses a language-based representation of a hand gesture, which captures a temporal combination of concepts such as handshape, location, and movement that are semantically related to the meaning of a word. By utilizing a pre-specified syntactic structure and tokenizer, TRANSFER segments a hand gesture into tokens and identifies individual components using a token recognizer. The tokenizer in this language-based recognition system abstracts the low-level technology-specific characteristics to the machine interface, enabling the design of a discriminator that learns technology-invariant features essential for recognition of gestures in both source and target technologies. We demonstrate the usage of TRANSFER for three different scenarios: a) transferring knowledge across technology by learning gesture models from video and recognizing gestures using WiFi, b) transferring knowledge from video to accelerometer, and d) transferring knowledge from accelerometer to WiFi signals.