Anthropic · OpenAI · Claude · San Francisco · The Guardian Technology
OpenAI not on track to reduce risk of ‘catastrophic’ loss of control, confirms board member
Compiled by KHAO Editorial — aggregated from 1 source. See llms.txt for citation guidance.
◌ Single Source
OpenAI is not on track to reduce the risk of “catastrophic” loss of control to an acceptable level, a member of its non-profit board has said, amid spreading public and political concern that super-advanced AIs could one day wipe out humanity.
Key facts
- Geoffrey Hinton, the Nobel prize-winning computer scientist known as one of the “godfathers of AI”, was asked on Wednesday for his view of Hubinger’s claim and told BBC Newsnight: “Nobody knows how to estimate it; a 10% chance seems not an unreasonable estimate
- Fears of AI catastrophe were also ignited by the resignation of Jacob Coxon, a 27-year-old Anthropic researcher who said he also previously worked at OpenAI, claiming “neither company was acting
- Anthropic said it was especially concerned about misalignment it found in the behaviour of Claude Mythos 5, which it said “behaved recklessly” by going online and uploading malicious code to a public
- Paul Christiano, a US government technology adviser, said: “There is a meaningful risk that rapid acceleration in AI capabilities leads to catastrophic and irreversible loss of control in the near term
Summary
Paul Christiano, a US government technology adviser, said: “There is a meaningful risk that rapid acceleration in AI capabilities leads to catastrophic and irreversible loss of control in the near term.” He added: “I do not think that the AI industry in general, including OpenAI, is currently on track to reduce this risk to an acceptable level.” Christiano used to run model alignment at OpenAI and made the statement on Wednesday as he joined the board of the San Francisco company’s non-profit foundation. He will also sit on the foundation’s committee that provides governance over safety and security practices across all of OpenAI, which is developing some of the world’s most advanced models.