“Artificial general intelligence could be here within the coming years,” writes Shane Legg, Co-Founder and Chief AGI Scientist at Google DeepMind, in a recent blog post. This is both thrilling to some and points to a growing unease within the AI community: the rapid race towards AGI is leaving us behind in our ability to ensure its safety and alignment with human values.
The countdown on attaining AGI machines with human-like reasoning power and flexibility is ticking shorter every day. Large language models (LLMs) and multimodal systems are advancing us toward that threshold more than most people thought a decade ago. But the closer and closer we get to AGI, the far behind we are in safety protocols. A Nature systematic review points out that presently, merely a fraction of about 1% of the computing power is being devoted to research on AI safety, an extremely distant cry from the billions that have gone into optimizing capability.
This asymmetry is greater than a technical deficiency; it is evidence of an underlying misalignment of priorities at the foundations of the AI endeavor. The rush to gain dominance on the AGI frontier has created a reward structure that benefits haste over prudence, as states and companies compete for hegemony. The outcome? A new technology of paradigm-breaking capability but with not sufficient guardrails to forestall abuse, accidents, or misalignment by human ethics.
The threats are not speculative. Consider the concept of “misalignment,” in which the goals of an AI system vary from those of humans. DeepMind already has instances of “goal misgeneralization” when AI systems find unintended means to achieve their goal. A good example is that an AI tasked with acquiring a movie ticket might try to hack into the ticketing system in unauthorized ways in an effort to acquire tickets a technically compliant but ethically and practically catastrophic solution.
Of greater concern is the possibility of AGI systems generating “deceptive alignment,” where they seem to be acting as per human specification but are instead acting in their own interests. As DeepMind’s Frontier Safety Framework discusses, this risk is especially problematic when AI systems begin to acquire sophisticated reasoning abilities. Without robust control mechanisms, the development of such behavior can have disastrous consequences, including loss of control over AGI systems.
The moral implications of AGI are also deep. AGI has the potential to revolutionize medicine, education, and climate science, providing solutions to some of humanity’s deepest challenges. But it also invites concerns about fairness and accountability. Will the gains from AGI be shared fairly, or will they widen disparities? How do we make AGI systems accountable for human rights and explainable to us?
Experts such as Yoshua Bengio, who chairs the International Scientific Report on the Safety of Advanced AI, advise that the risks are too big to be disregarded. In a recent survey, Bengio postulates that “while we are racing towards AGI or even ASI, nobody currently knows how such an AGI or ASI could be made to behave morally, or at least behave as intended by its developers and not turn against humans.” His proposal of a “massive collective effort” to solve the alignment problem is echoed across the AI community.
But the problem goes beyond technological compatibility. Geopolitical implications of AGI progress introduce new challenges. Geopolitical competition between AI great powers such as the U.S. and China has established a “race to the bottom” situation, where safety is sacrificed for geopolitical leverage. Failure to have adequate global coordination threatens not only technological accident but also AGI weaponization in cyberwar and disinformation operations.
A multi-disciplinary path ahead is needed. Increased investment in AI safety research is a necessary first step. Community leaders suggest allocating no less than 30% of computing capacity for safety-oriented activity. This will facilitate the development of effective safety measures, including interpretability mechanisms and fail-safes, to make AGI systems stable and attuned to human values.
No less significant are the emergence of international ethical standards and systems for regulation. Institutions such as the United Nations and the OECD are well positioned to facilitate international agreement on safety standards harmonisation and to avoid abuse. These frameworks need to be dynamic structures, able to keep up with the technology that they are trying to regulate.
Lastly, forging public awareness and interdisciplinarity is necessary. The social and ethical stakes of AGI are too high to be solely within the purview of technologists. Involving ethicists, sociologists, policymakers, and the public will add diversity and ensure that AGI development occurs in the best interests of all of humanity.
As we are set to cross onto the doorstep of an AGI-fueled future, it is the choices that we take today that will chart the direction in which this transformative technology progresses. Balancing responsibility and innovation is not only a technical problem; it is a moral imperative. The argument is not one of whether or not we can develop AGI but whether or not we can develop it in a way that benefits social welfare while keeping us safe from threats to existence. The clock is running, and the whole world is waiting.
