Musk’s sole witness at OpenAI trial warns of looming AGI arms race

When should we take AI doomers seriously?

This question lies at the heart of Elon Musk’s effort to restrict OpenAI’s for-profit AI operations. His legal team argues that the organization was originally established as a charity dedicated to AI safety but strayed from its mission in pursuit of profit. To support this claim, they referenced early emails and statements from the founders, who emphasized the need for a public-interest counterbalance to Google DeepMind.

Today, the defense presented its sole expert witness: Peter Russell, a computer science professor at the University of California, Berkeley, with decades of experience in AI research. His role was to provide context on AI and demonstrate that the technology poses significant risks worthy of concern.

Russell was among the signatories of an open letter in March 2023 calling for a six-month pause on AI research. In a notable contradiction, Musk also signed this same letter while simultaneously launching xAI, his own for-profit AI laboratory.

Russell testified before the jury and Judge Yvonne Gonzalez Rodgers that developing AI carries various risks, including cybersecurity threats, misalignment issues, and the winner-take-all dynamics of Artificial General Intelligence (AGI). He concluded that there is an inherent tension between pursuing AGI and ensuring safety.

Russell’s broader concerns about the existential threats of unregulated AI were not fully presented in open court after OpenAI’s attorneys objected, leading the judge to limit his testimony. However, Russell has long criticized the competitive arms race among global frontier labs racing to achieve AGI first, advocating for stricter government regulation of the field.

OpenAI’s legal team used their cross-examination to establish that Russell was not directly assessing the organization’s corporate structure or specific safety protocols.

Nevertheless, this reporter, along with the judge and jurors, must determine how much weight to give to the relationship between corporate profit motives and AI safety concerns. Almost all OpenAI founders have strongly warned about AI risks while simultaneously highlighting its benefits, striving to build AI as rapidly as possible, and developing plans for for-profit AI ventures they would control.

From an external perspective, a clear issue emerges: OpenAI’s founders realized after its inception that the organization required significantly more computing power to succeed. This funding could only come from for-profit investors. The founding team’s fear of AGI falling into the hands of a single entity drove them to seek capital, which ultimately fractured the team, created the current competitive landscape, and led to this lawsuit.

This same dynamic is unfolding at the national level: Senator Bernie Sanders’ proposal for a law imposing a moratorium on data center construction cites AI fears expressed by Musk, Sam Altman, Geoffrey Hinton, and others. Hoden Omar, who works at the Center for Data Innovation, a trade organization, objected to Sanders citing these fears without acknowledging their hopes, telling TechCrunch that “it is unclear why the public should discount everything tech billionaires say except when their words can be recruited to fill gaps in a precarious argument.”

Now, both sides of the case are asking the court to do exactly that: take parts of Altman and Musk’s arguments seriously, while discounting the parts less useful to their legal positions.