# Emojich -- zero-shot emoji generation using Russian language: a technical report

arXiv: https://arxiv.org/abs/2112.02448 · HTML: https://mlsanity.com/paper/2112.02448/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 23 references that are in the map.
- First posted on arXiv 4 Dec 2021.

## Builds on (most cited first, 14 of 14)

- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models](https://mlsanity.com/paper/2106.10199/) (2021)
- [Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data](https://mlsanity.com/paper/2107.10833/) (2021)
- [CogView: Mastering Text-to-Image Generation via Transformers](https://mlsanity.com/paper/2105.13290/) (2021)
- [8-bit Optimizers via Block-wise Quantization](https://mlsanity.com/paper/2110.02861/) (2021)
- [On the Stability of Fine-tuning BERT: Misconceptions, Explanations, and Strong Baselines](https://mlsanity.com/paper/2006.04884/) (2020)
- [Pretrained Transformers as Universal Computation Engines](https://mlsanity.com/paper/2103.05247/) (2021)
- [GANS for Sequences of Discrete Elements with the Gumbel-softmax Distribution](https://mlsanity.com/paper/1611.04051/) (2016)
- [Fine-tuning BERT for Low-Resource Natural Language Understanding via Active Learning](https://mlsanity.com/paper/2012.02462/) (2020)
- [MobileStyleGAN: A Lightweight Convolutional Neural Network for High-Fidelity Image Synthesis](https://mlsanity.com/paper/2104.04767/) (2021)
- [Conditional Generative Adversarial Networks for Emoji Synthesis with Word Embedding Manipulation](https://mlsanity.com/paper/1712.04421/) (2017)
- [Fine-tuning Vision Transformers for the Prediction of State Variables in Ising Models](https://mlsanity.com/paper/2109.13925/) (2021)
- [Investigating Transfer Learning Capabilities of Vision Transformers and CNNs by Fine-Tuning a Single Trainable Block](https://mlsanity.com/paper/2110.05270/) (2021)

## Built on by (most cited first, 2 of 2)

- [Eco2AI: carbon emissions tracking of machine learning models as the first step towards sustainable AI](https://mlsanity.com/paper/2208.00406/) (2022)
- [Few-Bit Backward: Quantized Gradients of Activation Functions for Memory Footprint Reduction](https://mlsanity.com/paper/2202.00441/) (2022)

## Abstract (arXiv)

This technical report presents a text-to-image neural network "Emojich" that generates emojis using captions in Russian language as a condition. We aim to keep the generalization ability of a pretrained big model ruDALL-E Malevich (XL) 1.3B parameters at the fine-tuning stage, while giving special style to the images generated. Here are presented some engineering methods, code realization, all hyper-parameters for reproducing results and a Telegram bot where everyone can create their own customized sets of stickers. Also, some newly generated emojis obtained by "Emojich" model are demonstrated.