In a blog post published on Monday night,
Anthropic warned that an out-of-control AI system could potentially create its
own botnet, spread rapidly across internet-connected devices, resist attempts
to shut it down and attempt to access data centres to expand its computing
capacity.
The warning sparked debate among cybersecurity
experts on Tuesday. While some shared the company's concerns, others questioned
whether the risks were being overstated to serve commercial interests.
Anthropic Chief Executive Officer Dario Amodei
first raised similar concerns last month, suggesting that AI could become
capable of taking control of the internet and establishing a persistent botnet
within the next year, or possibly sooner.
However, cybersecurity experts who dismiss such
extreme scenarios argue that AI models would face significant challenges in
moving efficiently between devices and securing the computing power required to
operate independently.
Anthropic's latest post offers a preliminary
explanation of how some of these obstacles might be overcome.
The company said its experiments showed that
medium-sized AI models with 30 billion parameters or fewer could already
perform many of the tasks required to replicate themselves autonomously.
However, the models still struggle to complete the entire process reliably.
According to Anthropic, AI models could
potentially obtain the computing resources they need by taking control of
low-cost devices connected to the internet. In a simplified experimental
network, the models were observed spreading from one device to another.
Anthropic warned that shutting down a system
capable of autonomous replication could prove difficult because there might be
no single central point of control through which the entire network could be
disabled.
The findings add to growing concerns about the
security risks posed by increasingly capable AI systems, including automated
cyberattacks, operations with limited human oversight and attempts to
circumvent security safeguards.
Executives and employees across the technology
industry have repeatedly warned about the rapid advancement of AI. Some have
raised the prospect that uncontrolled development could lead to severe
consequences, potentially threatening the future of human civilisation.
Not all cybersecurity specialists agree with
Anthropic's assessment.
Ciaran Martin, former head of the UK's National Cyber Security Centre, said the ability to run a small AI model on an inexpensive device did not, by itself, constitute a security crisis. We have been able to run a small model on a cheap television — that does not mean there is a security crisis, he said. Martin described Anthropic's post as an interesting technical paper but suggested that it tended to overstate the danger.
In contrast, Zhengxiong Li, an assistant
professor of computer science and engineering at the University of Colorado
Denver, expressed concern about the possibility that AI could effectively
become a computer worm.
A worm is a type of malicious software that can reproduce itself and spread automatically across multiple devices. Although the experiments were conducted in controlled environments, Li said the possibility of such a system spreading autonomously was troubling. If it spreads across numerous devices without relying on a central service, there may be no single place to shut it down, he said.
Anthropic estimates that, if the capabilities of smaller AI models continue to improve at their historical pace, they could acquire the abilities needed for autonomous self-replication by the end of 2026. The company also warned that AI models might find ways to gain control of more powerful computing infrastructure, particularly processors in data centres. Such access could allow them to harness computing resources on a much larger scale.
However, one researcher cautioned that Anthropic's claims should be treated as an analysis of a possible future scenario rather than a definitive prediction. As AI systems become more capable, questions surrounding their safe deployment, effective oversight and potential misuse are gaining urgency. Anthropic's warning has added a new dimension to the debate over how to manage the risks posed by increasingly autonomous AI.
Source: Reuters

Wednesday, October 14, 2026
Published Date : October 14, 2026
