Large AI Models Have a Prioritization Problem: Policy Implications and Solutions

Artificial intelligence (AI) models trained on large corpora must prioritize some information over others. Distilling vast data into simple user-friendly representations can lead to a prioritization problem, in which large AI models neglect crucial information in their outputs. This prioritization problem manifests for newsfeed-ranking algorithms and generative AI such as large language models (LLMs). This paper discusses two flavors of this problem: (1) models trained to prioritize coherence or engagement may inadvertently prioritize misleading information over accurate information, and (2) models trained to prioritize prevalent information will underrepresent the heterogeneity of voices, reflecting cultural perspectives that dominate the training data. The prioritization problem can critically undermine knowledge, but it is not inevitable. Policymakers can address the prioritization problem by modifying the training material, training objectives, institutional safeguards, and messaging around large AI models.