Small artificial intelligence models could soon be able to freely replicate across internet devices, forming botnets that do not require centralized computers to control them, according to an Anthropic Frontier Red Team paper.
The paper’s authors explored if foundation models could be run on unknown cheap devices and execute the actions required to replicate across other interconnected devices.
Anthropic bought 15 cheap Android TV boxes for $55 each to see how large an open-weight model could be before it could no longer be hosted on such devices.
The team found that a model with 26 billion parameters could generate 0.39 tokens per second. They also found that a 9 billion parameter model could run on 8 out of 9 of the different hardware types they tried.
The team then tried to see if a model could replicate across 8 emulated Android devices attached to a network. To make matters easier for the would-be-attackers, the researchers made the hacking process trivial; they only had to focus on the artificial intelligence processes.
Only the GLM-5.2 among the open-weight models was able to fully complete the replication process, doing so on four out of five tries, and managed to infect all 8 devices on the network.
Anthropic warned that the experiments should not be considered an accurate representation of how such an attack might unfold in the real world, but the results nevertheless show that attacks are becoming increasingly likely as models grow more capable.
Models with over 30 billion parameters do not have the ability to fully perform all actions required to replicate, but they can manage single aspects of the process. If development proceeds at its historical rate, then models will be able to achieve the capability demonstrated by GLM-5.2 by 2026. Self-replicating AI models could follow soon after.
The researchers also estimated the scale of such an attack.
If there was a Kimwolf-class botnet that had already successfully infected over 2 million devices, using up all 15 of the TV boxes mentioned earlier would result in 133000 instances of the 26-billion-parameter model being created. This would amount to a combined performance of up to 40 Nvidia H100 GPUs, but it would take more than a day for all instances to be created.
Such an attack surface is hard to prevent because there are no singular points of failure, but even if such a worm managed to successfully infect a network of devices, it would still be very visible to most network infrastructure.
Large traffic volumes and unusual communications patterns between devices that would otherwise not communicate with each other will raise flags with most internet service providers.
Anthropic also found that manufacturers could reduce the threat surface by replacing default passwords with more complex ones. Many previous botnets, including the infamous Mirai malware, were nearly impossible to stop because many IoT devices had default passwords that were too simple to guess.
Anthropic warned that while a fully autonomous worm was not yet available, a continuing increase in model size and capability, along with the growing ability of models to apply knowledge between different model classes, will severely limit the time defenders have to fix vulnerabilities in Internet of Things devices.
Read the original article: