An AI researcher at Anthropic quit the AI lab, criticising the approach of both Anthropic and OpenAI towards safety.
He outlined the potential risks of AI systems that are progressing so quickly that they could pose dangers to humans worldwide.
Jacob Coxon posted that he carried out pretraining research at both OpenAI and Anthropic, and that neither company was acting “responsibly” while “gambling with our lives” in an effort to reach self-improving superintelligence.
Mr.
Coxon’s resignation was also confirmed by The Wall Street Journal.
“The people building AI earnestly believe that it could kill us all by the end of the decade.
This is not a marketing stunt.
If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible - but I hear the same people express fear privately.
No other human activity poses this level of danger,” posted Mr.
Coxon on September 9.
U.S. to tell partners they must pick sides in AI race with China In his posts, he went on to outline the risks of rapidly progressing AI models that could hack into any system, gather power, and transform entire fields of knowledge.
Some solutions he proposed included pacing agreements between U.S. labs, though he acknowledged that international agreements to slow down AI development would be more difficult to arrive at.
However, Mr.
Coxon was backed up by Anthropic’s Alignment Science lead Evan Hubinger, who not only agreed with Mr.
Coxon but shared in the belief that AI could kill all humans.
He presented this outcome as a possibility in the future, even as critics accused both him and Mr.
Coxon of spreading fear-mongering rhetoric about AI.
“I personally think it is >10% within the next decade.
I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to,” posted Mr.
Hubinger, adding that the danger was linked to future models rather than those that are available presently.
