# Imitation Learning for Fashion Style Based on Hierarchical Multimodal Representation

arXiv: https://arxiv.org/abs/2004.06229 · HTML: https://mlsanity.com/paper/2004.06229/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 19 of its 49 references that are in the map.
- First posted on arXiv 13 Apr 2020.

## Placed in

- Offline RL and imitation learning from data / [Imitation learning methods](https://mlsanity.com/offlinerl/imitation-learning-methods/)

## Builds on (most cited first, 19 of 19)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks](https://mlsanity.com/paper/1511.06434/) (2015)
- [Generative Adversarial Imitation Learning](https://mlsanity.com/paper/1606.03476/) (2016)
- [Learning Robust Rewards with Adversarial Inverse Reinforcement Learning](https://mlsanity.com/paper/1710.11248/) (2017)
- [Supervised Learning of Universal Sentence Representations from Natural Language Inference Data](https://mlsanity.com/paper/1705.02364/) (2017)
- [Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimization](https://mlsanity.com/paper/1603.00448/) (2016)
- [Multimodal Generative Models for Scalable Weakly-Supervised Learning](https://mlsanity.com/paper/1802.05335/) (2018)
- [A Connection between Generative Adversarial Networks, Inverse Reinforcement Learning, and Energy-Based Models](https://mlsanity.com/paper/1611.03852/) (2016)
- [Learning Fashion Compatibility with Bidirectional LSTMs](https://mlsanity.com/paper/1707.05691/) (2017)
- [Generalized Product of Experts for Automatic and Principled Fusion of Gaussian Process Predictions](https://mlsanity.com/paper/1410.7827/) (2014)
- [POG: Personalized Outfit Generation for Fashion Recommendation at Alibaba iFashion](https://mlsanity.com/paper/1905.01866/) (2019)
- [Learning Visual Clothing Style with Heterogeneous Dyadic Co-occurrences](https://mlsanity.com/paper/1509.07473/) (2015)
- [Fashion Forward: Forecasting Visual Style in Fashion](https://mlsanity.com/paper/1705.06394/) (2017)
- [Creating Capsule Wardrobes from Fashion Images](https://mlsanity.com/paper/1712.02662/) (2017)
- [Learning the Latent "Look": Unsupervised Discovery of a Style-Coherent Embedding from Fashion Images](https://mlsanity.com/paper/1707.03376/) (2017)
- [Multi-Task Curriculum Transfer Deep Learning of Clothing Attributes](https://mlsanity.com/paper/1610.03670/) (2016)
- [Mining Fashion Outfit Composition Using An End-to-End Deep Learning Approach on Set Data](https://mlsanity.com/paper/1608.03016/) (2016)
- [Compatibility Family Learning for Item Recommendation and Generation](https://mlsanity.com/paper/1712.01262/) (2017)
- [Automatic Attribute Discovery with Neural Activations](https://mlsanity.com/paper/1607.07262/) (2016)

## Abstract (arXiv)

Fashion is a complex social phenomenon. People follow fashion styles from demonstrations by experts or fashion icons. However, for machine agent, learning to imitate fashion experts from demonstrations can be challenging, especially for complex styles in environments with high-dimensional, multimodal observations. Most existing research regarding fashion outfit composition utilizes supervised learning methods to mimic the behaviors of style icons. These methods suffer from distribution shift: because the agent greedily imitates some given outfit demonstrations, it can drift away from one style to another styles given subtle differences. In this work, we propose an adversarial inverse reinforcement learning formulation to recover reward functions based on hierarchical multimodal representation (HM-AIRL) during the imitation process. The hierarchical joint representation can more comprehensively model the expert composited outfit demonstrations to recover the reward function. We demonstrate that the proposed HM-AIRL model is able to recover reward functions that are robust to changes in multimodal observations, enabling us to learn policies under significant variation between different styles.