mlsanityData to 7 Oct 2026

Compression of end-to-end non-autoregressive image-to-speech system for low-resourced devices

arXiv 2312.00174 ↗First posted 30 Nov 2023eess.AS cs.AI cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

People with visual impairments have difficulty accessing touchscreen-enabled personal computing devices like mobile phones and laptops. The image-to-speech (ITS) systems can assist them in mitigating this problem, but their huge model size makes it extremely hard to be deployed on low-resourced embedded devices. In this paper, we aim to overcome this challenge by developing an efficient endto-end neural architecture for generating audio from tiny segments of display content on low-resource devices. We introduced a vision transformers-based image encoder and utilized knowledge distillation to compress the model from 6.1 million to 2.46 million parameters. Human and automatic evaluation results show that our approach leads to a very minimal drop in performance and can speed up the inference time by 22%.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 23 of its 26 references that are in the map. First posted on arXiv 30 Nov 2023.

Builds on

23 of 26 references.

References · 23

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  3. 2020End-to-End Object Detection with Transformerscited by 7,384
  4. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  5. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  6. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,250
  7. 2019TinyBERT: Distilling BERT for Natural Language Understandingcited by 1,153
  8. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,139
  9. 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 710
  10. 2020MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devicescited by 476
  11. 2016Synthetic Data for Text Localisation in Natural Imagescited by 396
  12. 2019Q8BERT: Quantized 8Bit BERTcited by 296
11 more
  1. 2018TED-LIUM 3: twice as much data and corpus repartition for experiments on speaker adaptationcited by 224
  2. 2014Synthetic Data and Artificial Neural Networks for Natural Scene Text Recognitioncited by 214
  3. 2021Vision Transformer for Fast and Efficient Scene Text Recognitioncited by 81
  4. 2021PortaSpeech: Portable and High-Quality Generative Text-to-Speechcited by 47
  5. 2022VQTTS: High-Fidelity Text-to-Speech Synthesis with Self-Supervised VQ Acoustic Featurecited by 32
  6. 2022JETS: Jointly Training FastSpeech2 and HiFi-GAN for End to End Text to Speechcited by 26
  7. 2019HBONet: Harmonious Bottleneck on Two Orthogonal Dimensionscited by 13
  8. 2022Source-Filter HiFi-GAN: Fast and Pitch Controllable High-Fidelity Neural Vocodercited by 11
  9. 2020Compact Speaker Embedding: lrx-vectorcited by 2
  10. 2023An End-to-End Neural Network for Image-to-Audio Transformationcited by 2
  11. 2023Fast and small footprint Hybrid HMM-HiFiGAN based system for speech synthesis in Indian languagescited by 1

Built on by

Not cited yet.