Anthropic warns small AI models may soon self-replicate on cheap smart devices

What Anthropic reported

Anthropic said on Monday, 12 October, that its experiments suggest small AI models may be months away from being capable enough to self-replicate on commercial internet-of-things (IoT) devices such as smart ovens and off-brand Android TV boxes. The research comes from the company's Frontier Red Team, which tests AI models for risky behaviour. The team studied whether a model could find vulnerable devices, install itself on them, run there and keep spreading without human help. The company focused on devices such as Android TVs, smart home appliances and other connected gadgets, which it said are often not protected well enough. The report examined multiple US- and Chinese-developed AI models.

Test results

The researchers found that open models with 30 billion parameters or fewer can already manage many of the individual replication steps, but they still make mistakes when trying to complete the whole process in sequence. To show that current hardware is already capable, the team bought 15 off-brand Android TV boxes for about $55 each. It split Google's open model Gemma4-26B across the devices and ran it, though it worked slowly at less than half a token per second. In a simplified test in which the hacking step was made easy, GLM-5.2, a larger open model, managed to spread from start to finish across a small network of emulated Android devices. Anthropic said that no current model proved both powerful and small enough to replicate reliably on a smart TV during testing.

Timing and open questions

Anthropic said that if small models keep improving at their current pace, they could reach autonomous replication by the end of 2026. The company also suggested that such systems would likely copy themselves slowly, relying on computing resources that are widely available and relatively poorly secured. Anthropic and its chief rival OpenAI have called on the international community to coordinate a slowdown in AI development to minimise catastrophic risks. That suggestion has had little buy-in from the United States and China, the two leading AI powers. Anthropic did not respond to Al Jazeera's questions, sent outside regular business hours, about the safety implications of the findings.

Containment and detection

The replicators would likely spread slowly, taking a day or more for each copy, and even a network as large as Kimwolf would have only about the computing power of 40 high-end Nvidia H100 chips. Anthropic warned that such a worm would be hard to stop because there is no central off switch. Unlike traditional malware, it could work out how to run on devices it has never encountered. Defenders also have advantages, since a spreading model would send large files between devices that do not normally communicate, which internet providers could detect. The team said providers could quarantine infected customers, and manufacturers should ship devices with stronger security by default.

Home and small business IoT devices might be the first target for such attacks because they're connected to the internet and, historically, have had minimal security measures in place.

Background on compromised devices

Cheap smart devices have already been exploited at scale. The Kimwolf botnet infected over two million devices, mostly off-brand Android TV boxes, and a Chinese state-linked network took control of 1,500 routers and devices. Anthropic's researchers also anticipated a future awash with agents running on all types of hardware, including some that aren't under human control. Anthropic argued that IoT devices have been vulnerable for a long time, but lightweight agents may be far more willing than human attackers to put in the effort to compromise large fractions of the world's IoT computing.