# Recent Advances in Generative AI and Large Language Models: Current Status, Challenges, and Perspectives

arXiv: https://arxiv.org/abs/2407.14962 · HTML: https://mlsanity.com/paper/2407.14962/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 108 of its 182 references that are in the map.
- First posted on arXiv 20 Jul 2024.

## Builds on (most cited first, 30 of 108)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [Parameter-Efficient Transfer Learning for NLP](https://mlsanity.com/paper/1902.00751/) (2019)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer](https://mlsanity.com/paper/1701.06538/) (2017)

## Built on by (most cited first, 30 of 30)

- [Enabling Novel Mission Operations and Interactions with ROSA: The Robot Operating System Agent](https://mlsanity.com/paper/2410.06472/) (2024)
- [IFAR: Multi-Perspective and Multi-Level Causal Discovery with LLMs](https://mlsanity.com/paper/2409.05559/) (2024)
- [Scaling Up Biomedical Vision-Language Models: Fine-Tuning, Instruction Tuning, and Multi-Modal Learning](https://mlsanity.com/paper/2505.17436/) (2025)
- [Advancing Talking Head Generation: A Comprehensive Survey of Multi-Modal Methodologies, Datasets, Evaluation Metrics, and Loss Functions](https://mlsanity.com/paper/2507.02900/) (2025)
- [Who Owns the Output? Bridging Law and Technology in LLMs Attribution](https://mlsanity.com/paper/2504.01032/) (2025)
- [HarmNet: A Framework for Adaptive Multi-Turn Jailbreak Attacks on Large Language Models](https://mlsanity.com/paper/2510.18728/) (2025)
- [Combining TSL and LLM to Automate REST API Testing: A Comparative Study](https://mlsanity.com/paper/2509.05540/) (2025)
- [Multi-Modal Data-Enhanced Foundation Models for Prediction and Control in Wireless Networks: A Survey](https://mlsanity.com/paper/2601.03181/) (2026)
- [Generative AI and Large Language Models in Language Preservation: Opportunities and Challenges](https://mlsanity.com/paper/2501.11496/) (2025)
- [Artificial Intelligence for Atmospheric Sciences: A Research Roadmap](https://mlsanity.com/paper/2506.16281/) (2025)
- [Agentic Satellite-Augmented Low-Altitude Economy and Terrestrial Networks: A Survey on Generative Approaches](https://mlsanity.com/paper/2507.14633/) (2025)
- [Generative Multi-Objective Bayesian Optimization with Scalable Batch Evaluations for Sample-Efficient De Novo Molecular Design](https://mlsanity.com/paper/2512.17659/) (2025)
- [Ethical Concerns of Generative AI and Mitigation Strategies: A Systematic Mapping Study](https://mlsanity.com/paper/2502.00015/) (2025)
- [ReaCritic: Reasoning Transformer-based DRL Critic-model Scaling For Wireless Networks](https://mlsanity.com/paper/2505.10992/) (2025)
- [LLM-Powered Quantum Code Transpilation](https://mlsanity.com/paper/2507.12480/) (2025)
- [Towards Inclusive NLP: Assessing Compressed Multilingual Transformers across Diverse Language Benchmarks](https://mlsanity.com/paper/2507.19699/) (2025)
- [Beyond Benchmarks: LLM Evaluation with an Anthropomorphic and Lifecycle-oriented Roadmap](https://mlsanity.com/paper/2508.18646/) (2025)
- [A Survey: Towards Privacy and Security in Mobile Large Language Models](https://mlsanity.com/paper/2509.02411/) (2025)
- [NL2GenSym: Natural Language to Generative Symbolic Rules for SOAR Cognitive Architecture via Large Language Models](https://mlsanity.com/paper/2510.09355/) (2025)
- [Federated Attention: A Distributed Paradigm for Collaborative LLM Inference over Edge Networks](https://mlsanity.com/paper/2511.02647/) (2025)
- [Semimage: HSV-Based Semantic Image Encoding for Disentangled Text Representation](https://mlsanity.com/paper/2512.00088/) (2025)
- [Is Our Chatbot Telling Lies? Assessing Correctness of an LLM-based Dutch Support Chatbot](https://mlsanity.com/paper/2411.00034/) (2024)
- [Retrospective: A CORDIC Based Configurable Activation Function for NN Applications](https://mlsanity.com/paper/2503.14354/) (2025)
- [Physical foundations for trustworthy medical imaging: a review for artificial intelligence researchers](https://mlsanity.com/paper/2505.02843/) (2025)
- [Psychology-Driven Enhancement of Humour Translation](https://mlsanity.com/paper/2507.09259/) (2025)
- [CALR: Corrective Adaptive Low-Rank Decomposition for Efficient Large Language Model Layer Compression](https://mlsanity.com/paper/2508.16680/) (2025)
- [Impact and Implications of Generative AI for Enterprise Architects in Agile Environments: A Systematic Literature Review](https://mlsanity.com/paper/2510.22003/) (2025)
- [Co-evolving Agent Architectures and Interpretable Reasoning for Automated Optimization](https://mlsanity.com/paper/2604.17708/) (2026)
- [Agentic and Generative AI for Open-Source Intelligence and Cyber Investigations: Taxonomy, Evaluation, Challenges, and Future Directions](https://mlsanity.com/paper/2607.03233/) (2026)
- [HyperZip: Efficient Data Compression through Personalized Diffusion LLMs with Hypernetworks](https://mlsanity.com/paper/2609.36357/) (2026)

## Abstract (arXiv)

The emergence of Generative Artificial Intelligence (AI) and Large Language Models (LLMs) has marked a new era of Natural Language Processing (NLP), introducing unprecedented capabilities that are revolutionizing various domains. This paper explores the current state of these cutting-edge technologies, demonstrating their remarkable advancements and wide-ranging applications. Our paper contributes to providing a holistic perspective on the technical foundations, practical applications, and emerging challenges within the evolving landscape of Generative AI and LLMs. We believe that understanding the generative capabilities of AI systems and the specific context of LLMs is crucial for researchers, practitioners, and policymakers to collaboratively shape the responsible and ethical integration of these technologies into various domains. Furthermore, we identify and address main research gaps, providing valuable insights to guide future research endeavors within the AI research community.