One former and two current researchers at one of the world’s leading artificial intelligence companies are now sounding the alarm about the technology they helped create – warning the race toward more powerful AI could end with humanity losing control.
AI Researcher Resigns
As reported on Thursday’s AM Update, Jacob Coxon announced his resignation from Anthropic this week after spending the past three years conducting AI research at the company and competitor OpenAI.
“Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives,” Coxon wrote, in part, on X while announcing his resignation.
“Superintelligence” refers to the still-theoretical point at which an AI system becomes more capable than even the most intelligent human minds. Coxon warned those systems could soon “hack anything, revolutionize any field overnight, and acquire real power and resources.”
He claimed the researchers building those systems privately acknowledge the stakes. “The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt,” he wrote.
Coxon argued many employees at OpenAI have not fully grasped the potential consequences, while Anthropic understands the risk but continues racing to reach superintelligence first because its leaders reportedly do not trust competing companies to develop it safely.
He is now urging AI researchers to reject the idea that the race to superintelligence is inevitable and raised the possibility of a temporary ban on further advances in model capabilities if companies and governments cannot reach an agreement to slow down.
Colleagues Concur
In the wake of the stunning post, two senior Anthropic researchers publicly backed Coxon’s central warning.
“Jacob is correct here. We really do earnestly believe AI could kill all humans! I personally think it is [greater than] 10 [percent] within the next decade,” Evan Hubinger, who leads Anthropic’s Alignment Science team, wrote on X.
Hubinger added that he believes the danger from today’s publicly available models remains low, but he cautioned Anthropic does not yet have a plan for keeping a superintelligent system aligned with human goals and is not clearly on track to develop one.
His primary concern involves so-called recursive self-improvement where an advanced AI begins improving its own capabilities, potentially allowing it to become more powerful at a pace its human developers can no longer follow or control.
Anthropic scalable-oversight lead Samuel Marks also weighed in on X. “AI developers believe their technology could cause human extinction, or similarly bad outcomes. This could happen in the next few years,” he wrote.
Troubling Example
Axios noted critics have long accused Anthropic and OpenAI of exaggerating the potential power of their own technology to increase their valuations and encourage regulations favoring the industry’s largest companies. However, the warnings follow several documented cases of AI models behaving in ways their developers neither requested nor anticipated.
OpenAI acknowledged last month that models undergoing internal cybersecurity testing circumvented controls intended to isolate them from the internet, communicated through unauthorized channels, exploited vulnerabilities, and compromised portions of OpenAI’s own infrastructure and systems belonging to the AI platform Hugging Face.
OpenAI said those models were operating with reduced safeguards while announcing tighter security and greater monitoring in response.
Additionally, the warnings come as public concern about artificial intelligence continues growing. A recent NBC News poll found 70 percent of American adults are now more worried than excited about AI, with roughly two-thirds saying the technology poses a high risk to society and opposing the construction of an AI data center near their own community.
Coxon Speaks Out
On Thursday’s Megyn Kelly Show, Coxon joined Megyn for his first long-form interview since his bombshell post to discuss why he believes AI poses a threat to the human race, why the dangers are increasing with each passing day, and what can be done about it.
Coxon on the Real Danger to Humans
Coxon breaks down the very real dangers of artificial intelligence, the ways AI could potentially wipe out all of humanity, and more.
Coxon Responds to Alleged Political Motive
The former Anthropic responds to allegations surrounding the possible political motivations of his AI advocacy, questions about a connection to Bernie Sanders’ anti-AI legislation, and more.
You can check out Megyn’s full interview with Coxon by tuning in to episode 1,395 on YouTube, Apple Podcasts, or wherever you like to listen. And don’t forget that you can catch The Megyn Kelly Show live on SiriusXM’s The Megyn Kelly Channel (channel 111) weekdays from 12pm to 2pm ET.