An FNet based Auto Encoder for Long Sequence News Story Generation
Not on the map. Why?
Abstract (arXiv)
In this paper, we design an auto encoder based off of Google's FNet Architecture in order to generate text from a subset of news stories contained in Google's C4 dataset. We discuss previous attempts and methods to generate text from autoencoders and non LLM Models. FNET poses multiple advantages to BERT based encoders in the realm of efficiency which train 80% faster on GPUs and 70% faster on TPUs. We then compare outputs of how this autencoder perfroms on different epochs. Finally, we analyze what outputs the encoder produces with different seed text.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 4 of its 20 references that are in the map. First posted on arXiv 15 Nov 2022.
Builds on
4 of 20 references.
References · 4
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
- 2022GPT-NeoX-20B: An Open-Source Autoregressive Language Modelcited by 713
- 2020A Survey of Convolutional Neural Networks: Analysis, Applications, and Prospectscited by 397
- 2021FNet: Mixing Tokens with Fourier Transformscited by 347
Built on by
Not cited yet.