# The Solution for the AIGC Inference Performance Optimization Competition

arXiv: https://arxiv.org/abs/2407.04991 · HTML: https://mlsanity.com/paper/2407.04991/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 33 references that are in the map.
- First posted on arXiv 6 Jul 2024.

## Builds on (most cited first, 11 of 11)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Rethinking the Value of Network Pruning](https://mlsanity.com/paper/1810.05270/) (2018)
- [Fast Transformers with Clustered Attention](https://mlsanity.com/paper/2007.04825/) (2020)
- [Fast WordPiece Tokenization](https://mlsanity.com/paper/2012.15524/) (2020)
- [Ultimate tensorization: compressing convolutional and FC layers alike](https://mlsanity.com/paper/1611.03214/) (2016)
- [ERNIE-ViLG: Unified Generative Pre-training for Bidirectional Vision-Language Generation](https://mlsanity.com/paper/2112.15283/) (2021)
- [Semi-Supervised Multi-Modal Multi-Instance Multi-Label Deep Network with Optimal Transport](https://mlsanity.com/paper/2104.08489/) (2021)
- [Learning Adaptive Embedding Considering Incremental Class](https://mlsanity.com/paper/2008.13351/) (2020)
- [Robust Semi-supervised Learning by Wisely Leveraging Open-set Data](https://mlsanity.com/paper/2405.06979/) (2024)
- [S2OSC: A Holistic Semi-Supervised Approach for Open Set Classification](https://mlsanity.com/paper/2008.04662/) (2020)
- [MAP: Model Aggregation and Personalization in Federated Learning with Incomplete Classes](https://mlsanity.com/paper/2404.09232/) (2024)

## Abstract (arXiv)

In recent years, the rapid advancement of large-scale pre-trained language models based on transformer architectures has revolutionized natural language processing tasks. Among these, ChatGPT has gained widespread popularity, demonstrating human-level conversational abilities and attracting over 100 million monthly users by late 2022. Concurrently, Baidu's commercial deployment of the Ernie Wenxin model has significantly enhanced marketing effectiveness through AI-driven technologies. This paper focuses on optimizing high-performance inference for Ernie models, emphasizing GPU acceleration and leveraging the Paddle inference framework. We employ techniques such as Faster Transformer for efficient model processing, embedding layer pruning to reduce computational overhead, and FP16 half-precision inference for enhanced computational efficiency. Additionally, our approach integrates efficient data handling strategies using multi-process parallel processing to minimize latency. Experimental results demonstrate that our optimized solution achieves up to an 8.96x improvement in inference speed compared to standard methods, while maintaining competitive performance.