Anthropic · OpenAI · Claude · CNBC Technology
Fears over the safety of AI systems, and their potential to wipe out humanity, gained new
Compiled by KHAO Editorial — aggregated from 1 source. See llms.txt for citation guidance.
◌ Single Source
Evan Hubinger, an alignment lead at Anthropic, said on X that he thinks there is more than a 10% chance that AI could kill all humans within the next decade, after a colleague quit over safety fears.
Key facts
- Mistral hit a $24 billion valuation as Samsung led a $3.5 billion funding round for the French startup
- Google said it would invest at least $15 billion into AI infrastructure in Finland, marking the tech giant's largest single investment in Europe
- Qualcomm issued warrants to Amazon to acquire $4 billion worth of the chipmaker's stock as part of an AI infrastructure deal
- Before Aidan Gomez took the top position at AI startup Cohere, he was one of the co-authors of the 2017 research paper Attention Is All You Need, better known as the Transformer paper
Summary
Fears over the safety of AI systems, and their potential to wipe out humanity, gained new, viral traction this week. But it was in Hubinger's reply to his own post that revealed where exactly his concerns lay. "What I am worried about is superintelligence arising from recursive self-improvement, as we have said is happening faster than we thought," he said. Recursive self-improvement, or RSI, is when AI itself helps improve the process of building new models, potentially leading to spiralling capability as better systems build better systems and so on.