# Aligning AI with Public Values: Deliberation and Decision-Making for Governing Multimodal LLMs in Political Video Analysis

arXiv: https://arxiv.org/abs/2410.01817 · HTML: https://mlsanity.com/paper/2410.01817/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 125 references that are in the map.
- First posted on arXiv 15 Sep 2024.

## Builds on (most cited first, 26 of 26)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrieval](https://mlsanity.com/paper/2104.00650/) (2021)
- [Model Cards for Model Reporting](https://mlsanity.com/paper/1810.03993/) (2018)
- [BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Dataset](https://mlsanity.com/paper/2307.04657/) (2023)
- [Whose Opinions Do Language Models Reflect?](https://mlsanity.com/paper/2303.17548/) (2023)
- [Localizing Moments in Video with Natural Language](https://mlsanity.com/paper/1708.01641/) (2017)
- [Open-Vocabulary Semantic Segmentation with Mask-adapted CLIP](https://mlsanity.com/paper/2210.04150/) (2022)
- [Multi-modal Transformer for Video Retrieval](https://mlsanity.com/paper/2007.10639/) (2020)
- [Expanding Language-Image Pretrained Models for General Video Recognition](https://mlsanity.com/paper/2208.02816/) (2022)
- [ActionCLIP: A New Paradigm for Video Action Recognition](https://mlsanity.com/paper/2109.08472/) (2021)
- [From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP Models](https://mlsanity.com/paper/2305.08283/) (2023)
- [Just Ask: Learning to Answer Questions from Millions of Narrated Videos](https://mlsanity.com/paper/2012.00451/) (2020)
- [Language-Grounded Indoor 3D Semantic Segmentation in the Wild](https://mlsanity.com/paper/2204.07761/) (2022)
- [Fine-tuned CLIP Models are Efficient Video Learners](https://mlsanity.com/paper/2212.03640/) (2022)
- [Revisiting the "Video" in Video-Language Understanding](https://mlsanity.com/paper/2206.01720/) (2022)
- [Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claims](https://mlsanity.com/paper/2004.07213/) (2020)
- [Bridging the Gap between Object and Image-level Representations for Open-Vocabulary Detection](https://mlsanity.com/paper/2207.03482/) (2022)
- [Opening up ChatGPT: Tracking openness, transparency, and accountability in instruction-tuned text generators](https://mlsanity.com/paper/2307.05532/) (2023)
- [Who Audits the Auditors? Recommendations from a field scan of the algorithmic auditing ecosystem](https://mlsanity.com/paper/2310.02521/) (2023)
- [VX2TEXT: End-to-End Learning of Video-Based Text Generation From Multimodal Inputs](https://mlsanity.com/paper/2101.12059/) (2021)
- [U.S. Public Opinion on the Governance of Artificial Intelligence](https://mlsanity.com/paper/1912.12835/) (2019)
- [Regulating Artificial Intelligence: Proposal for a Global Solution](https://mlsanity.com/paper/2005.11072/) (2020)
- [Unpacking How Decentralized Autonomous Organizations (DAOs) Work in Practice](https://mlsanity.com/paper/2304.09822/) (2023)
- [Crowdsourcing for Participatory Democracies: Efficient Elicitation of Social Choice Functions](https://mlsanity.com/paper/1406.7542/) (2014)
- [GDPR Compliant Blockchains-A Systematic Literature Review](https://mlsanity.com/paper/2104.00648/) (2021)
- [SoK: The Gap Between Data Rights Ideals and Reality](https://mlsanity.com/paper/2312.01511/) (2023)

## Built on by (most cited first, 7 of 7)

- [Hidden Persuaders: LLMs' Political Leaning and Their Influence on Voters](https://mlsanity.com/paper/2410.24190/) (2024)
- [A Multimodal, Multilingual, and Multidimensional Pipeline for Fine-grained Crowdsourcing Earthquake Damage Evaluation](https://mlsanity.com/paper/2506.03360/) (2025)
- [Signals of Provenance: Practices & Challenges of Navigating Indicators in AI-Generated Media for Sighted and Blind Individuals](https://mlsanity.com/paper/2505.16057/) (2025)
- [Personhood Credentials: Human-Centered Design Recommendation Balancing Security, Usability, and Trust](https://mlsanity.com/paper/2502.16375/) (2025)
- [PRAC3 (Privacy, Reputation, Accountability, Consent, Credit, Compensation): Long Tailed Risks of Voice Actors in AI Data-Economy](https://mlsanity.com/paper/2507.16247/) (2025)
- [Polyphony: Diffusion-based Dual-Hand Action Segmentation with Alternating Vision Transformer and Semantic Conditioning](https://mlsanity.com/paper/2605.31115/) (2026)
- [V.O.I.C.E (Voice, Ownership, Identity, Control, Expression): Risk Taxonomy of Synthetic Voice Generation From Empirical Data](https://mlsanity.com/paper/2604.24794/) (2026)

## Abstract (arXiv)

How AI models should deal with political topics has been discussed, but it remains challenging and requires better governance. This paper examines the governance of large language models through individual and collective deliberation, focusing on politically sensitive videos. We conducted a two-step study: interviews with 10 journalists established a baseline understanding of expert video interpretation; 114 individuals through deliberation using InclusiveAI, a platform that facilitates democratic decision-making through decentralized autonomous organization (DAO) mechanisms. Our findings reveal distinct differences in interpretative priorities: while experts emphasized emotion and narrative, the general public prioritized factual clarity, objectivity, and emotional neutrality. Furthermore, we examined how different governance mechanisms - quadratic vs. weighted voting and equal vs. 20/80 voting power - shape users' decision-making regarding AI behavior. Results indicate that voting methods significantly influence outcomes, with quadratic voting reinforcing perceptions of liberal democracy and political equality. Our study underscores the necessity of selecting appropriate governance mechanisms to better capture user perspectives and suggests decentralized AI governance as a potential way to facilitate broader public engagement in AI development, ensuring that varied perspectives meaningfully inform design decisions.