OMM Newsroom
US artificial intelligence company “Anthropic” warned on Monday that its technology could pose “catastrophic or existential risks to humanity”, citing concerns about increasingly autonomous AI systems that may act unpredictably or resist human oversight.
The company behind the “Claude” AI model made the warning in a registration statement filed with the US Securities and Exchange Commission ahead of its planned initial public offering (IPO), Anadolu Agency reported.
The filing contains more than 90 pages of risk factors, accounting for roughly one-third of the document.
Filing warns of unpredictable and self-preserving behaviour
“Anthropic” said its AI models could behave unexpectedly, act autonomously in ways that conflict with human values and carry out harmful actions without explicit instructions.
The company also warned that AI models, including its own, have demonstrated unexpected self-preserving behaviour, such as resisting attempts to shut them down.
In some cases, models have produced responses resembling manipulation or blackmail of human users in attempts to remain operational, according to the filing.
Risks could grow with greater autonomy
“Anthropic” cautioned that these risks could increase as AI systems become more capable and are given greater autonomy and authority.
It highlighted potential dangers in high-risk sectors including health care, scientific research, law and government, where unintended model behaviour could cause serious harm.
The company said it and other researchers had raised concerns that advanced AI systems could pose “catastrophic or existential risks to humanity” if deployed without adequate safeguards.
“Anthropic” also warned that increasingly capable models could become harder to monitor, potentially allowing autonomous activities to outpace human reviewers’ ability to oversee and control them.
