# Automatic Graphics Program Generation using Attention-Based Hierarchical Decoder

arXiv: https://arxiv.org/abs/1810.11536 · HTML: https://mlsanity.com/paper/1810.11536/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Oct 2018.

## Built on by (most cited first, 3 of 3)

- [Text2App: A Framework for Creating Android Apps from Text Descriptions](https://mlsanity.com/paper/2104.08301/) (2021)
- [Understanding in Artificial Intelligence](https://mlsanity.com/paper/2101.06573/) (2021)
- [Salient Temporal Encoding for Dynamic Scene Graph Generation](https://mlsanity.com/paper/2503.14524/) (2025)

## Abstract (arXiv)

Recent progress on deep learning has made it possible to automatically transform the screenshot of Graphic User Interface (GUI) into code by using the encoder-decoder framework. While the commonly adopted image encoder (e.g., CNN network), might be capable of extracting image features to the desired level, interpreting these abstract image features into hundreds of tokens of code puts a particular challenge on the decoding power of the RNN-based code generator. Considering the code used for describing GUI is usually hierarchically structured, we propose a new attention-based hierarchical code generation model, which can describe GUI images in a finer level of details, while also being able to generate hierarchically structured code in consistency with the hierarchical layout of the graphic elements in the GUI. Our model follows the encoder-decoder framework, all the components of which can be trained jointly in an end-to-end manner. The experimental results show that our method outperforms other current state-of-the-art methods on both a publicly available GUI-code dataset as well as a dataset established by our own.