# MetaGreen: Meta-Learning Inspired Transformer Selection for Green Semantic Communication

arXiv: https://arxiv.org/abs/2406.16962 · HTML: https://mlsanity.com/paper/2406.16962/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 45 references that are in the map.
- First posted on arXiv 22 Jun 2024.

## Builds on (most cited first, 25 of 25)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [OPT: Open Pre-trained Transformer Language Models](https://mlsanity.com/paper/2205.01068/) (2022)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference](https://mlsanity.com/paper/1712.05877/) (2017)
- [Scaling Autoregressive Models for Content-Rich Text-to-Image Generation](https://mlsanity.com/paper/2206.10789/) (2022)
- [Meta-Learning with Implicit Gradients](https://mlsanity.com/paper/1909.04630/) (2019)
- [The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits](https://mlsanity.com/paper/2402.17764/) (2024)
- [SDXL-Lightning: Progressive Adversarial Diffusion Distillation](https://mlsanity.com/paper/2402.13929/) (2024)
- [Model Complexity of Deep Learning: A Survey](https://mlsanity.com/paper/2103.05127/) (2021)
- [BitsFusion: 1.99 bits Weight Quantization of Diffusion Model](https://mlsanity.com/paper/2406.04333/) (2024)
- [Ps and Qs: Quantization-aware pruning for efficient low latency neural network inference](https://mlsanity.com/paper/2102.11289/) (2021)
- [LLM Comparator: Visual Analytics for Side-by-Side Evaluation of Large Language Models](https://mlsanity.com/paper/2402.10524/) (2024)
- [Wireless Multi-Agent Generative AI: From Connected Intelligence to Collective Intelligence](https://mlsanity.com/paper/2307.02757/) (2023)
- [Great Power, Great Responsibility: Recommendations for Reducing Energy for Training Language Models](https://mlsanity.com/paper/2205.09646/) (2022)
- [HULK: An Energy Efficiency Benchmark Platform for Responsible Natural Language Processing](https://mlsanity.com/paper/2002.05829/) (2020)
- [Green, Quantized Federated Learning over Wireless Networks: An Energy-Efficient Design](https://mlsanity.com/paper/2207.09387/) (2022)
- [Making Sense of Meaning: A Survey on Metrics for Semantic and Goal-Oriented Communication](https://mlsanity.com/paper/2303.10867/) (2023)
- [Energy Efficient Semantic Communication over Wireless Networks with Rate Splitting](https://mlsanity.com/paper/2301.01987/) (2023)
- [Energy-Aware Service Offloading for Semantic Communications in Wireless Networks](https://mlsanity.com/paper/2401.15924/) (2024)

## Abstract (arXiv)

Semantic Communication can transform the way we transmit information, prioritizing meaningful and effective content over individual symbols or bits. This evolution promises significant benefits, including reduced latency, lower bandwidth usage, and higher throughput compared to traditional communication. However, the development of Semantic Communication faces a crucial challenge: the need for universal metrics to benchmark the joint effects of semantic information loss and energy consumption. This research introduces an innovative solution: the ``Energy-Optimized Semantic Loss'' (EOSL) function, a novel multi-objective loss function that effectively balances semantic information loss and energy consumption. Through comprehensive experiments on transformer models, including energy benchmarking, we demonstrate the remarkable effectiveness of EOSL-based model selection. We have established that EOSL-based transformer model selection achieves up to 83% better similarity-to-power ratio (SPR) compared to BLEU score-based selection and 67% better SPR compared to solely lowest power usage-based selection. Furthermore, we extend the applicability of EOSL to diverse and varying contexts, inspired by the principles of Meta-Learning. By cumulatively applying EOSL, we enable the model selection system to adapt to this change, leveraging historical EOSL values to guide the learning process. This work lays the foundation for energy-efficient model selection and the development of green semantic communication.