Anthropic · Claude · China · Open Source · OpenAI · CBS News Technology
Anthropic confirmed it banned and posted accounts engaging in misuse of its AI and applied the findings to strengthen safeguards
Compiled by KHAO Editorial — aggregated from 2 sources. See llms.txt for citation guidance.
✓ KHAO Verified
But that kind of oversight is more difficult with "open-weight" models.
Key facts
- And there's evidence that these models are potentially highly accessible, Hugging Face lists more than 8,000 models under the search phrase "abliterated
- A model is like a helper," said NYU cybersecurity professor and Fulbright Iceland scholar Justin Cappos
- In a July letter, more than 70 companies including Google, Microsoft and NVIDIA said that open-weight models make advanced AI "more accessible" and urged policymakers not to prohibit them
- Last month, Mindgard could jailbreak Moonshot's Kimi K3 and 2.6, open-weight models currently available to consumers
Summary
The revelation last month that Claude was used "in ways that could support biological weapons development " was made possible, in part, because Anthropic's AI models operate on a closed system overseen by the company. Anthropic said it banned and reported accounts engaging in misuse of its AI and used the findings to strengthen safeguards. Claude's models are closed, so its weights can't be modified by users after being molded by the company. "A model is like a helper," said NYU cybersecurity professor and Fulbright Iceland scholar Justin Cappos.