# Self-Augmented Multi-Modal Feature Embedding

arXiv: https://arxiv.org/abs/2103.04731 · HTML: https://mlsanity.com/paper/2103.04731/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 22 references that are in the map.
- First posted on arXiv 8 Mar 2021.

## Builds on (most cited first, 13 of 13)

- [Conditional Generative Adversarial Nets](https://mlsanity.com/paper/1411.1784/) (2014)
- [Time Series Classification from Scratch with Deep Neural Networks: A Strong Baseline](https://mlsanity.com/paper/1611.06455/) (2016)
- [Gated Multimodal Units for Information Fusion](https://mlsanity.com/paper/1702.01992/) (2017)
- [LSTM Fully Convolutional Networks for Time Series Classification](https://mlsanity.com/paper/1709.05206/) (2017)
- [Cross-modal Deep Variational Hand Pose Estimation](https://mlsanity.com/paper/1803.11404/) (2018)
- [Imaging Time-Series to Improve Classification and Imputation](https://mlsanity.com/paper/1506.00327/) (2015)
- [Learning Cross-Modal Embeddings with Adversarial Networks for Cooking Recipes and Food Images](https://mlsanity.com/paper/1905.01273/) (2019)
- [CM-GANs: Cross-modal Generative Adversarial Networks for Common Representation Learning](https://mlsanity.com/paper/1710.05106/) (2017)
- [Multimodal Fusion with Deep Neural Networks for Audio-Video Emotion Recognition](https://mlsanity.com/paper/1907.03196/) (2019)
- [Multimodal Classification for Analysing Social Media](https://mlsanity.com/paper/1708.02099/) (2017)
- [Modality Conversion of Handwritten Patterns by Cross Variational Autoencoders](https://mlsanity.com/paper/1906.06142/) (2019)
- [Image-Text Multi-Modal Representation Learning by Adversarial Backpropagation](https://mlsanity.com/paper/1612.08354/) (2016)
- [Dynamic Weight Alignment for Temporal Convolutional Neural Networks](https://mlsanity.com/paper/1712.06530/) (2017)

## Built on by (most cited first, 1 of 1)

- [Dynamic Data Augmentation with Gating Networks for Time Series Recognition](https://mlsanity.com/paper/2111.03253/) (2021)

## Abstract (arXiv)

Oftentimes, patterns can be represented through different modalities. For example, leaf data can be in the form of images or contours. Handwritten characters can also be either online or offline. To exploit this fact, we propose the use of self-augmentation and combine it with multi-modal feature embedding. In order to take advantage of the complementary information from the different modalities, the self-augmented multi-modal feature embedding employs a shared feature space. Through experimental results on classification with online handwriting and leaf images, we demonstrate that the proposed method can create effective embeddings.