On the Trustworthiness of Generative Foundation Models: Guideline, Assessment, and Perspective
Not on the map. Why?
Abstract (arXiv)
Generative Foundation Models (GenFMs) have emerged as transformative tools. However, their widespread adoption raises critical concerns regarding trustworthiness across dimensions. This paper presents a comprehensive framework to address these challenges through three key contributions. First, we systematically review global AI governance laws and policies from governments and regulatory bodies, as well as industry practices and standards. Based on this analysis, we propose a set of guiding principles for GenFMs, developed through extensive multidisciplinary collaboration that integrates technical, ethical, legal, and societal perspectives. Second, we introduce TrustGen, the first dynamic benchmarking platform designed to evaluate trustworthiness across multiple dimensions and model types, including text-to-image, large language, and vision-language models. TrustGen leverages modular components--metadata curation, test case generation, and contextual variation--to enable adaptive and iterative assessments, overcoming the limitations of static evaluation methods. Using TrustGen, we reveal significant progress in trustworthiness while identifying persistent challenges. Finally, we provide an in-depth discussion of the challenges and future directions for trustworthy GenFMs, which reveals the complex, evolving nature of trustworthiness, highlighting the nuanced trade-offs between utility and trustworthiness, and consideration for various downstream applications, identifying persistent challenges and providing a strategic roadmap for future research. This work establishes a holistic framework for advancing trustworthiness in GenAI, paving the way for safer and more responsible integration of GenFMs into critical applications. To facilitate advancement in the community, we release the toolkit for dynamic evaluation.
Key facts. Cited by 41 papers in the map (data through 14 Oct 2026). First posted on arXiv 20 Feb 2025.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2025: 28; 2026: 13.
41 papers.
Sort:
By area:
- 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 116
- 2025One Token to Fool LLM-as-a-Judgecited by 60
- 2025Recent Advances in Large Langauge Model Benchmarks against Data Contamination: From Static to Dynamic Evaluationcited by 38
- 2025Building a Foundational Guardrail for General Agentic Systems via Synthetic Datacited by 29
- 2025ManipLVM-R1: Reinforcement Learning for Reasoning in Embodied Manipulation with Large Vision-Language Modelscited by 27
- 2025FaithCoT-Bench: Benchmarking Instance-Level Faithfulness of Chain-of-Thought Reasoningcited by 26
- 2025LangCoop: Collaborative Driving with Languagecited by 25
- 2025Audio Jailbreak: An Open Comprehensive Benchmark for Jailbreaking Large Audio-Language Modelscited by 21
- 2025Why LLM Safety Guardrails Collapse After Fine-tuning: A Similarity Analysis Between Alignment and Fine-tuning Datasetscited by 20
- 2025HumaniBench: A Human-Centric Framework for Large Multimodal Models Evaluationcited by 19
- 2025Dissecting Logical Reasoning in LLMs: A Fine-Grained Evaluation and Supervision Studycited by 17
- 2026ToolGate: Contract-Grounded and Verified Tool Execution for LLMscited by 15
29 more
- 2025SocialMaze: A Benchmark for Evaluating and Enhancing Social Reasoning in Large Language Models in Complex Social Environmentscited by 11
- 2025Privacy Reasoning in Ambiguous Contextscited by 11
- 2025Beyond Surface-Level Patterns: An Essence-Driven Defense Framework Against Jailbreak Attacks in LLMscited by 9
- 2026ProbeLLM: Automating Principled Diagnosis of LLM Failurescited by 9
- 2025Privacy-Aware Decoding: Mitigating Privacy Leakage of Large Language Models in Retrieval-Augmented Generationcited by 8
- 2025Beyond Accuracy: The Role of Calibration in Self-Improving Large Language Modelscited by 7
- 2025Generative AI for Autonomous Driving: Frontiers and Opportunitiescited by 7
- 2025Discovering Knowledge Deficiencies of Language Models on Massive Knowledge Basecited by 6
- 2025SUV: Scalable Large Language Model Copyright Compliance with Regularized Selective Unlearningcited by 5
- 2025Are We on the Right Way for Assessing Document Retrieval-Augmented Generation?cited by 5
- 2025Secure and Robust Watermarking for AI-generated Images: A Comprehensive Surveycited by 5
- 2025Cache Mechanism for Agent RAG Systemscited by 5
- 2026Emergent Risks in Generative Multi-Agent Systemscited by 5
- 2025Improving Large Language Models Function Calling and Interpretability via Guided-Structured Templatescited by 4
- 2026Detecting Unfaithful Chain-of-Thought via Circuit-Guided Internal-External Discrepancycited by 3
- 2025The Role of Open-Source LLMs in Shaping the Future of GeoAIcited by 2
- 2025SafeCoop: Unravelling Full Stack Safety in Agentic Collaborative Drivingcited by 2
- 2026Diagnosing and Repairing Unsafe Channels in Vision-Language Models via Causal Discovery and Dual-Modal Safety Subspace Projectioncited by 2
- 2025Leveraging Importance Sampling to Detach Alignment Modules from Large Language Modelscited by 1
- 2025When Personalization Tricks Detectors: The Feature-Inversion Trap in Machine-Generated Text Detectioncited by 1
- 2026Uncertainty-Aware Distribution-to-Distribution Flow Matching for Scientific Imagingcited by 1
- 2026Guardian-as-an-Advisor: Advancing Next-Generation Guardian Models for Trustworthy LLMscited by 1
- 2025Towards provable probabilistic safety for scalable embodied AI systemscited by 0
- 2026Fact-Checking with Large Language Models via Probabilistic Certainty and Consistencycited by 0
- 2026Alignment Risks from Capability-Seeking RL Trainingcited by 0
- 2026On Understanding, Identifying, and Mitigating Vulnerabilities in Agentic Large Language Modelscited by 0
- 2026A Proposal for an Agentic AI Architecture to Support Multi-Domain Decision-Making in the Brazilian Armed Forcescited by 0
- 2026SSP-Bench: A Hybrid Data Generation Framework for Safety, Security, and Privacy Evaluationcited by 0
- 2026VEX-Bench: Benchmarking Verification Complexity of LLM-Generated Misinformationcited by 0
- 2026VEX-Bench: Benchmarking Verification Complexity of LLM-Generated Misinformation28 Sep 2026
- 2026SSP-Bench: A Hybrid Data Generation Framework for Safety, Security, and Privacy Evaluation21 Sep 2026
- 2026A Proposal for an Agentic AI Architecture to Support Multi-Domain Decision-Making in the Brazilian Armed Forces17 Sep 2026
- 2026On Understanding, Identifying, and Mitigating Vulnerabilities in Agentic Large Language Models11 Aug 2026
- 2026Detecting Unfaithful Chain-of-Thought via Circuit-Guided Internal-External Discrepancy25 May 2026
- 2026Guardian-as-an-Advisor: Advancing Next-Generation Guardian Models for Trustworthy LLMs8 Apr 2026
- 2026Emergent Risks in Generative Multi-Agent Systems29 Mar 2026
- 2026Diagnosing and Repairing Unsafe Channels in Vision-Language Models via Causal Discovery and Dual-Modal Safety Subspace Projection28 Mar 2026
- 2026Uncertainty-Aware Distribution-to-Distribution Flow Matching for Scientific Imaging23 Mar 2026
- 2026ProbeLLM: Automating Principled Diagnosis of LLM Failures13 Feb 2026
- 2026Alignment Risks from Capability-Seeking RL Training12 Feb 2026
- 2026ToolGate: Contract-Grounded and Verified Tool Execution for LLMs8 Jan 2026
29 more
- 2026Fact-Checking with Large Language Models via Probabilistic Certainty and Consistency5 Jan 2026
- 2025Cache Mechanism for Agent RAG Systems4 Nov 2025
- 2025SafeCoop: Unravelling Full Stack Safety in Agentic Collaborative Driving20 Oct 2025
- 2025When Personalization Tricks Detectors: The Feature-Inversion Trap in Machine-Generated Text Detection14 Oct 2025
- 2025Building a Foundational Guardrail for General Agentic Systems via Synthetic Data10 Oct 2025
- 2025FaithCoT-Bench: Benchmarking Instance-Level Faithfulness of Chain-of-Thought Reasoning5 Oct 2025
- 2025Secure and Robust Watermarking for AI-generated Images: A Comprehensive Survey30 Sep 2025
- 2025Improving Large Language Models Function Calling and Interpretability via Guided-Structured Templates22 Sep 2025
- 2025Are We on the Right Way for Assessing Document Retrieval-Augmented Generation?5 Aug 2025
- 2025Privacy-Aware Decoding: Mitigating Privacy Leakage of Large Language Models in Retrieval-Augmented Generation5 Aug 2025
- 2025One Token to Fool LLM-as-a-Judge11 Jul 2025
- 2025Privacy Reasoning in Ambiguous Contexts13 Jun 2025
- 2025Why LLM Safety Guardrails Collapse After Fine-tuning: A Similarity Analysis Between Alignment and Fine-tuning Datasets5 Jun 2025
- 2025Towards provable probabilistic safety for scalable embodied AI systems5 Jun 2025
- 2025Dissecting Logical Reasoning in LLMs: A Fine-Grained Evaluation and Supervision Study5 Jun 2025
- 2025SocialMaze: A Benchmark for Evaluating and Enhancing Social Reasoning in Large Language Models in Complex Social Environments29 May 2025
- 2025Leveraging Importance Sampling to Detach Alignment Modules from Large Language Models26 May 2025
- 2025ManipLVM-R1: Reinforcement Learning for Reasoning in Embodied Manipulation with Large Vision-Language Models22 May 2025
- 2025Audio Jailbreak: An Open Comprehensive Benchmark for Jailbreaking Large Audio-Language Models21 May 2025
- 2025HumaniBench: A Human-Centric Framework for Large Multimodal Models Evaluation16 May 2025
- 2025Generative AI for Autonomous Driving: Frontiers and Opportunities13 May 2025
- 2025The Role of Open-Source LLMs in Shaping the Future of GeoAI24 Apr 2025
- 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deployment22 Apr 2025
- 2025LangCoop: Collaborative Driving with Language18 Apr 2025
- 2025Beyond Accuracy: The Role of Calibration in Self-Improving Large Language Models3 Apr 2025
- 2025Discovering Knowledge Deficiencies of Language Models on Massive Knowledge Base30 Mar 2025
- 2025SUV: Scalable Large Language Model Copyright Compliance with Regularized Selective Unlearning29 Mar 2025
- 2025Beyond Surface-Level Patterns: An Essence-Driven Defense Framework Against Jailbreak Attacks in LLMs26 Feb 2025
- 2025Recent Advances in Large Langauge Model Benchmarks against Data Contamination: From Static to Dynamic Evaluation23 Feb 2025
- 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 116
- 2025ManipLVM-R1: Reinforcement Learning for Reasoning in Embodied Manipulation with Large Vision-Language Modelscited by 27
- 2025FaithCoT-Bench: Benchmarking Instance-Level Faithfulness of Chain-of-Thought Reasoningcited by 26
- 2025Dissecting Logical Reasoning in LLMs: A Fine-Grained Evaluation and Supervision Studycited by 17
- 2025SocialMaze: A Benchmark for Evaluating and Enhancing Social Reasoning in Large Language Models in Complex Social Environmentscited by 11
- 2025Privacy Reasoning in Ambiguous Contextscited by 11
- 2025Improving Large Language Models Function Calling and Interpretability via Guided-Structured Templatescited by 4
- 2026Detecting Unfaithful Chain-of-Thought via Circuit-Guided Internal-External Discrepancycited by 3
- 2026Detecting Unfaithful Chain-of-Thought via Circuit-Guided Internal-External Discrepancy25 May 2026
- 2025FaithCoT-Bench: Benchmarking Instance-Level Faithfulness of Chain-of-Thought Reasoning5 Oct 2025
- 2025Improving Large Language Models Function Calling and Interpretability via Guided-Structured Templates22 Sep 2025
- 2025Privacy Reasoning in Ambiguous Contexts13 Jun 2025
- 2025Dissecting Logical Reasoning in LLMs: A Fine-Grained Evaluation and Supervision Study5 Jun 2025
- 2025SocialMaze: A Benchmark for Evaluating and Enhancing Social Reasoning in Large Language Models in Complex Social Environments29 May 2025
- 2025ManipLVM-R1: Reinforcement Learning for Reasoning in Embodied Manipulation with Large Vision-Language Models22 May 2025
- 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deployment22 Apr 2025
- 2025Building a Foundational Guardrail for General Agentic Systems via Synthetic Datacited by 29
- 2026ToolGate: Contract-Grounded and Verified Tool Execution for LLMscited by 15
- 2026On Understanding, Identifying, and Mitigating Vulnerabilities in Agentic Large Language Modelscited by 0
- 2026A Proposal for an Agentic AI Architecture to Support Multi-Domain Decision-Making in the Brazilian Armed Forcescited by 0
- 2026A Proposal for an Agentic AI Architecture to Support Multi-Domain Decision-Making in the Brazilian Armed Forces17 Sep 2026
- 2026On Understanding, Identifying, and Mitigating Vulnerabilities in Agentic Large Language Models11 Aug 2026
- 2026ToolGate: Contract-Grounded and Verified Tool Execution for LLMs8 Jan 2026
- 2025Building a Foundational Guardrail for General Agentic Systems via Synthetic Data10 Oct 2025
- 2025One Token to Fool LLM-as-a-Judgecited by 60
- 2025Why LLM Safety Guardrails Collapse After Fine-tuning: A Similarity Analysis Between Alignment and Fine-tuning Datasetscited by 20
- 2025SUV: Scalable Large Language Model Copyright Compliance with Regularized Selective Unlearningcited by 5
- 2025Leveraging Importance Sampling to Detach Alignment Modules from Large Language Modelscited by 1
- 2025One Token to Fool LLM-as-a-Judge11 Jul 2025
- 2025Why LLM Safety Guardrails Collapse After Fine-tuning: A Similarity Analysis Between Alignment and Fine-tuning Datasets5 Jun 2025
- 2025Leveraging Importance Sampling to Detach Alignment Modules from Large Language Models26 May 2025
- 2025SUV: Scalable Large Language Model Copyright Compliance with Regularized Selective Unlearning29 Mar 2025
- 2025Audio Jailbreak: An Open Comprehensive Benchmark for Jailbreaking Large Audio-Language Modelscited by 21
- 2025Beyond Surface-Level Patterns: An Essence-Driven Defense Framework Against Jailbreak Attacks in LLMscited by 9
- 2025Secure and Robust Watermarking for AI-generated Images: A Comprehensive Surveycited by 5
- 2026Alignment Risks from Capability-Seeking RL Trainingcited by 0
- 2026Alignment Risks from Capability-Seeking RL Training12 Feb 2026
- 2025Secure and Robust Watermarking for AI-generated Images: A Comprehensive Survey30 Sep 2025
- 2025Audio Jailbreak: An Open Comprehensive Benchmark for Jailbreaking Large Audio-Language Models21 May 2025
- 2025Beyond Surface-Level Patterns: An Essence-Driven Defense Framework Against Jailbreak Attacks in LLMs26 Feb 2025
- 2025HumaniBench: A Human-Centric Framework for Large Multimodal Models Evaluationcited by 19
- 2025Are We on the Right Way for Assessing Document Retrieval-Augmented Generation?cited by 5
- 2026Diagnosing and Repairing Unsafe Channels in Vision-Language Models via Causal Discovery and Dual-Modal Safety Subspace Projectioncited by 2
- 2026Diagnosing and Repairing Unsafe Channels in Vision-Language Models via Causal Discovery and Dual-Modal Safety Subspace Projection28 Mar 2026
- 2025Are We on the Right Way for Assessing Document Retrieval-Augmented Generation?5 Aug 2025
- 2025HumaniBench: A Human-Centric Framework for Large Multimodal Models Evaluation16 May 2025
- 2025Recent Advances in Large Langauge Model Benchmarks against Data Contamination: From Static to Dynamic Evaluationcited by 38
- 2026ProbeLLM: Automating Principled Diagnosis of LLM Failurescited by 9
- 2025Discovering Knowledge Deficiencies of Language Models on Massive Knowledge Basecited by 6
- 2026Emergent Risks in Generative Multi-Agent Systemscited by 5
- 2025The Role of Open-Source LLMs in Shaping the Future of GeoAIcited by 2
- 2025When Personalization Tricks Detectors: The Feature-Inversion Trap in Machine-Generated Text Detectioncited by 1
- 2026Guardian-as-an-Advisor: Advancing Next-Generation Guardian Models for Trustworthy LLMscited by 1
- 2025Towards provable probabilistic safety for scalable embodied AI systemscited by 0
- 2026Fact-Checking with Large Language Models via Probabilistic Certainty and Consistencycited by 0
- 2026SSP-Bench: A Hybrid Data Generation Framework for Safety, Security, and Privacy Evaluationcited by 0
- 2026VEX-Bench: Benchmarking Verification Complexity of LLM-Generated Misinformationcited by 0
- 2026VEX-Bench: Benchmarking Verification Complexity of LLM-Generated Misinformation28 Sep 2026
- 2026SSP-Bench: A Hybrid Data Generation Framework for Safety, Security, and Privacy Evaluation21 Sep 2026
- 2026Guardian-as-an-Advisor: Advancing Next-Generation Guardian Models for Trustworthy LLMs8 Apr 2026
- 2026Emergent Risks in Generative Multi-Agent Systems29 Mar 2026
- 2026ProbeLLM: Automating Principled Diagnosis of LLM Failures13 Feb 2026
- 2026Fact-Checking with Large Language Models via Probabilistic Certainty and Consistency5 Jan 2026
- 2025When Personalization Tricks Detectors: The Feature-Inversion Trap in Machine-Generated Text Detection14 Oct 2025
- 2025Towards provable probabilistic safety for scalable embodied AI systems5 Jun 2025
- 2025The Role of Open-Source LLMs in Shaping the Future of GeoAI24 Apr 2025
- 2025Discovering Knowledge Deficiencies of Language Models on Massive Knowledge Base30 Mar 2025
- 2025Recent Advances in Large Langauge Model Benchmarks against Data Contamination: From Static to Dynamic Evaluation23 Feb 2025