Cross-lingual Adaptation for Recipe Retrieval with Mixup
Not on the map. Why?
Abstract (arXiv)
Cross-modal recipe retrieval has attracted research attention in recent years, thanks to the availability of large-scale paired data for training. Nevertheless, obtaining adequate recipe-image pairs covering the majority of cuisines for supervised learning is difficult if not impossible. By transferring knowledge learnt from a data-rich cuisine to a data-scarce cuisine, domain adaptation sheds light on this practical problem. Nevertheless, existing works assume recipes in source and target domains are mostly originated from the same cuisine and written in the same language. This paper studies unsupervised domain adaptation for image-to-recipe retrieval, where recipes in source and target domains are in different languages. Moreover, only recipes are available for training in the target domain. A novel recipe mixup method is proposed to learn transferable embedding features between the two domains. Specifically, recipe mixup produces mixed recipes to form an intermediate domain by discretely exchanging the section(s) between source and target recipes. To bridge the domain gap, recipe mixup loss is proposed to enforce the intermediate domain to locate in the shortest geodesic path between source and target domains in the recipe embedding space. By using Recipe 1M dataset as source domain (English) and Vireo-FoodTransfer dataset as target domain (Chinese), empirical experiments verify the effectiveness of recipe mixup for cross-lingual adaptation in the context of image-to-recipe retrieval.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 29 of its 54 references that are in the map. First posted on arXiv 8 May 2022.
Builds on
29 of 54 references.
References · 29
- 2017Attention Is All You Needcited by 59,932
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2014Adam: A Method for Stochastic Optimizationcited by 36,960
- 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
- 2015Domain-Adversarial Training of Neural Networkscited by 3,023
- 2016Enriching Word Vectors with Subword Informationcited by 1,578
- 2015Learning Transferable Features with Deep Adaptation Networkscited by 1,323
- 2017Adversarial Discriminative Domain Adaptationcited by 1,309
- 2017CyCADA: Cycle-Consistent Adversarial Domain Adaptationcited by 1,079
- 2014Deep Domain Confusion: Maximizing for Domain Invariancecited by 597
- 2019Adversarial Domain Adaptation with Domain Mixupcited by 250
17 more
- 2020How Does Mixup Help With Robustness and Generalization?cited by 193
- 2019Cross-Lingual Ability of Multilingual BERT: An Empirical Studycited by 182
- 2018DLOW: Domain Flow for Adaptation and Generalizationcited by 176
- 2018Partial Adversarial Domain Adaptationcited by 127
- 2020The Tatoeba Translation Challenge -- Realistic Data Sets for Low Resource and Multilingual MTcited by 127
- 2018Importance Weighted Adversarial Nets for Partial Domain Adaptationcited by 90
- 2021Token Shift Transformer for Video Classificationcited by 61
- 2021IDM: An Intermediate Domain Module for Domain Adaptive Person Re-IDcited by 51
- 2016Adversarial Deep Averaging Networks for Cross-Lingual Sentiment Classificationcited by 48
- 2019Learning Cross-Modal Embeddings with Adversarial Networks for Cooking Recipes and Food Imagescited by 38
- 2018Cross-Modal Retrieval in the Cooking Context: Learning Semantic Text-Image Embeddingscited by 35
- 2021Revamping Cross-Modal Recipe Retrieval with Hierarchical Transformers and Self-supervised Learningcited by 28
- 2019MULE: Multimodal Universal Language Embeddingcited by 22
- 2019Virtual Mixup Training for Unsupervised Domain Adaptationcited by 21
- 2019Food Recommendation: Framework, Existing Solutions and Challengescited by 21
- 2020MCEN: Bridging Cross-Modal Gap between Cooking Recipes and Dish Images with Latent Variable Modelcited by 16
- 2017MHTN: Modal-adversarial Hybrid Transfer Network for Cross-modal Retrievalcited by 12
Built on by
Citing papers per year: 2023: 1; 2024: 0; 2025: 2.
3 papers.