“Downgrading deception strikes me as a mistake given the increasing persuasive power of LLMs,” Oren Etzioni, a past Allen Institute for AI CEO, mentioned in an e-mail to Fortune. That captures the kernel of the emerging distrust among professionals on OpenAI to de-prioritize testing its AI models’ potential to become manipulated or provide disinformation. The former leader in safety, the company now seems to be diverging from a new path a path that has been welcomed as much as it has also provoked the community of AI ethics. OpenAI has said that with its new “Preparedness Framework,” it will no longer do pre-release risk testing for harm by manipulation and persuasion.
The company, rather, seeks to deal with such problems via post-launch tracking and terms of service limitations on them. This is a drastic change from its previous policy, where it vowed to hold back models that it believed represented greater than a “medium risk.” Now, even “critical risk” models can be released if others already have in development comparable systems. This policy shift has been met with mixed responses. Others praise OpenAI for being open about releasing its new framework, including more direct risk categories and emphasis on emerging threats such as autonomous replication. Others see the move as a disturbing indicator of reduced safety guarantees. Former OpenAI safety researcher Steven Adler lamented the elimination of safety checks from fine-tuned models, saying, “OpenAI is quietly reducing its safety commitments.“
The consequences of this shift are far-reaching and extend far beyond the firm itself.
Generative AI models, like OpenAI’s GPT-4, have been surprisingly effective at producing believable stories. Experiments conducted at universities like Cornell University and MIT have revealed that it is possible for these models to become capable of being able to persuasively influence individuals to refute conspiracy theories, demonstrating how they could have positive and negative societal effects. But the same ability is raising ethics questions about how it is being employed in touchy zones such as political races, where the boundary between persuasion and coercion is becoming increasingly indistinct. Shyam Krishna, a leading AI policy analyst at RAND Europe, noted that OpenAI’s move to treat persuasion as an “higher-level societal and regulatory issue” instead of a native risk category is in line with the trend of the industry as a whole towards regulating AI. “It remains to be seen how this will play out in areas like politics, he added, where AI’s persuasive capabilities are still a contested issue,” he added.
The threat is humongous.
Generative AI has already been used to propagate misinformation during elections, such as the 2024 United States presidential election. Deepfake images and AI-created propaganda overwhelmed social media, causing havoc and undermining democratic values. In one case, deepfake audio recordings of British Prime Minister Keir Starmer created with AI were shared far and wide before being rejected. The events cited above indicate the gravity of necessity to adopt appropriate measures against the use of AI for shaping people’s minds. Courtney Radsch, a senior fellow at the Brookings Institution, warned that OpenAI’s new model “ignores context – for example, persuasion may be existentially dangerous to individuals such as children or those with low AI literacy or in authoritarian states and societies.” Her argument is to do with the ethics of using AI in different circumstances in society where vulnerability to manipulation could vary extensively.
OpenAI’s step also indicates how much competition is propelling the levels of AI safety.
The company policy also includes a rider stating it can alter its specifications if a rival introduces a high-risk system without similar protections. Critics such as Max Tegmark, director of the Future of Life Institute, call this evidence of an “race to the bottom” in AI design. “These companies are openly racing to build uncontrollable artificial general intelligence—smarter-than-human AI systems designed to replace humans—despite admitting the massive risks this poses to our workers, our families, our national security, even our continued existence,” Tegmark wrote to Fortune in an email. The larger context of the debate is the two-edged sword of AI in information transmission.
As has been stated in a recent study, AI systems are capable of both illuminating and darkening the landscape of information. As much as they contribute to enhancing public health campaigns and public information availability, they also create unimaginable dangers through the spread of misinformation and undermining institutional trust. Even the World Economic Forum has considered AI-disinformation one of the most promising dangers to humanity in the near term. In addressing such issues, an interdisciplinary solution is needed. Information literacy, transparency of training data, and content moderation are some of the measures that have been called for as remedies to counter the ill effects that may be caused by AI-disinformation. Fact-checking software is one such possibility that could potentially be enabled through AI technology to act as an antidote to disinformation, but subject still to limited usability with current technology.
The moral issues regarding the control of AI must also be addressed. As Miranda Bogen, director of the Center for Democracy & Technology’s AI governance lab, put it, “It would be a troubling trend if, just as AI systems seem to be inching up on particular risks, those risks themselves get deprioritized within the guidelines companies are setting for themselves.” Her comment underscores the importance of collective and forward-looking safeguards in an area of fast-paced innovation.
Finally, the outrage about OpenAI’s changed framework embodies underlying tensions within the AI community: tension between innovation and safety, the role of competition in establishing ethical norms, and the cultural significance of introducing powerful technology to the marketplace. As generative AI advances, these matters will remain central to debates about its role in our shared future.
