mlsanityData to 7 Oct 2026

PaLM 2 Technical Report

arXiv 2305.10403 ↗First posted 17 May 2023cs.CL cs.AIcited by 1,154

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

We introduce PaLM 2, a new state-of-the-art language model that has better multilingual and reasoning capabilities and is more compute-efficient than its predecessor PaLM. PaLM 2 is a Transformer-based model trained using a mixture of objectives. Through extensive evaluations on English and multilingual language, and reasoning tasks, we demonstrate that PaLM 2 has significantly improved quality on downstream tasks across different model sizes, while simultaneously exhibiting faster and more efficient inference compared to PaLM. This improved efficiency enables broader deployment while also allowing the model to respond faster, for a more natural pace of interaction. PaLM 2 demonstrates robust reasoning capabilities exemplified by large improvements over PaLM on BIG-Bench and other reasoning tasks. PaLM 2 exhibits stable performance on a suite of responsible AI evaluations, and enables inference-time control over toxicity without additional overhead or impact on other capabilities. Overall, PaLM 2 achieves state-of-the-art performance across a diverse set of tasks and capabilities. When discussing the PaLM 2 family, it is important to distinguish between pre-trained models (of various sizes), fine-tuned variants of these models, and the user-facing products that use these models. In particular, user-facing products typically include additional pre- and post-processing steps. Additionally, the underlying models may evolve over time. Therefore, one should not expect the performance of user-facing products to exactly match the results reported in this report.

Key facts. Cited by 1,154 papers in the map (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 115 of its 167 references that are in the map. First posted on arXiv 17 May 2023.

Builds on

115 of 167 references.

References · 115

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2022Training language models to follow instructions with human feedbackcited by 15,712
  5. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  6. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  7. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  8. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  9. 2020Scaling Laws for Neural Language Modelscited by 6,411
  10. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  11. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  12. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
38 more
  1. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  2. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  3. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  4. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  5. 2021Program Synthesis with Large Language Modelscited by 3,509
  6. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  7. 2022Training Compute-Optimal Large Language Modelscited by 2,927
  8. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  9. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,818
  10. 2020Extracting Training Data from Large Language Modelscited by 2,144
  11. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,114
  12. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  13. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
  14. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,813
  15. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,793
  16. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,767
  17. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,718
  18. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,538
  19. 2018Datasheets for Datasetscited by 1,371
  20. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,359
  21. 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,308
  22. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,235
  23. 2022LaMDA: Language Models for Dialog Applicationscited by 1,218
  24. 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,196
  25. 2018Model Cards for Model Reportingcited by 1,068
  26. 2016The LAMBADA dataset: Word prediction requiring a broad discourse contextcited by 960
  27. 2021Ethical and social risks of harm from Language Modelscited by 957
  28. 2020Language (Technology) is Power: A Critical Survey of "Bias" in NLPcited by 951
  29. 2021GLaM: Efficient Scaling of Language Models with Mixture-of-Expertscited by 890
  30. 2019DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphscited by 883
  31. 2017RACE: Large-scale ReAding Comprehension Dataset From Examinationscited by 851
  32. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 847
  33. 2019Adversarial NLI: A New Benchmark for Natural Language Understandingcited by 798
  34. 2022Quantifying Memorization Across Neural Language Modelscited by 755
  35. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 736
  36. 2023The Flan Collection: Designing Data and Methods for Effective Instruction Tuningcited by 734
  37. 2020How Much Knowledge Can You Pack Into the Parameters of a Language Model?cited by 713
  38. 2021Deduplicating Training Data Makes Language Models Bettercited by 705

50 most cited of 115 All 115 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 379; 2024: 514; 2025: 204; 2026: 57.

1,154 papers.

Sort:

By area:

  1. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  2. 2023Qwen Technical Reportcited by 3,273
  3. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
  4. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,928
  5. 2025Gemma 3 Technical Reportcited by 1,686
  6. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,597
  7. 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,385
  8. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,326
  9. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,221
  10. 2023AgentBench: Evaluating LLMs as Agentscited by 1,211
  11. 2023Instruction-Following Evaluation for Large Language Modelscited by 1,143
  12. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,124
38 more
  1. 2024Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Predictioncited by 1,005
  2. 2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 965
  3. 2023Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotationscited by 895
  4. 2024Gemma: Open Models Based on Gemini Research and Technologycited by 889
  5. 2023Reasoning with Language Model is Planning with World Modelcited by 846
  6. 2023Large Language Models as Optimizerscited by 830
  7. 2024Autoregressive Model Beats Diffusion: Llama for Scalable Image Generationcited by 760
  8. 2023WizardCoder: Empowering Code Large Language Models with Evol-Instructcited by 728
  9. 2024Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMscited by 719
  10. 2023Aligning Large Multimodal Models with Factually Augmented RLHFcited by 703
  11. 2024SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilitiescited by 679
  12. 2023The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)cited by 677
  13. 2023Baichuan 2: Open Large-scale Language Modelscited by 664
  14. 2023A Comprehensive Overview of Large Language Modelscited by 639
  15. 2023Language Model Beats Diffusion -- Tokenizer is Key to Visual Generationcited by 633
  16. 2023Safe RLHF: Safe Reinforcement Learning from Human Feedbackcited by 630
  17. 2023WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instructcited by 618
  18. 2023AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviorscited by 580
  19. 2023Textbooks Are All You Needcited by 561
  20. 2024Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Modelscited by 552
  21. 2023The Falcon Series of Open Language Modelscited by 526
  22. 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Promptscited by 520
  23. 2024Measuring Multimodal Mathematical Reasoning with MATH-Vision Datasetcited by 516
  24. 2024Large Language Models: A Surveycited by 506
  25. 2023Large Language Models Are Zero-Shot Time Series Forecasterscited by 491
  26. 2023MAmmoTH: Building Math Generalist Models through Hybrid Instruction Tuningcited by 483
  27. 2023VideoPoet: A Large Language Model for Zero-Shot Video Generationcited by 479
  28. 2023"Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Modelscited by 457
  29. 2023LMSYS-Chat-1M: A Large-Scale Real-World LLM Conversation Datasetcited by 449
  30. 2023Learning Interactive Real-World Simulatorscited by 440
  31. 2023Catastrophic Jailbreak of Open-source LLMs via Exploiting Generationcited by 439
  32. 2023Scaling Relationship on Learning Mathematical Reasoning with Large Language Modelscited by 417
  33. 2023How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resourcescited by 405
  34. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 381
  35. 2024Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Datacited by 371
  36. 2023Editing Large Language Models: Problems, Methods, and Opportunitiescited by 367
  37. 2023A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesiscited by 350
  38. 2023Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflictscited by 333

50 most cited of 1,154 All 1,154 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025From System 1 to System 2: A Survey of Reasoning Large Language Models
  2. 2024Tandem Transformers for Inference Efficient LLMs
  3. 2023Palm: Predicting Actions through Language Models @ Ego4D Long-Term Action Anticipation Challenge 2023
  4. 2023PaLM-E: An Embodied Multimodal Language Model
  5. 2025ARM2: Adaptive Reasoning Model with Vision Understanding and Executable Code
  6. 2024LLM2: Let Large Language Models Harness System 2 Reasoning
  7. 2026Yuan3.0 Flash: An Open Multimodal Large Language Model for Enterprise Applications
  8. 2026PaLMR: Towards Faithful Visual Reasoning via Multimodal Process Alignment