The tech world is abuzz with a stark warning from Anthropic, the AI safety lab founded by former OpenAI researchers. In a paper published this week, they argue that the current trajectory of AI development could lead to systems that operate entirely without human oversight, a scenario they call 'human-free AI'. This is not science fiction. It is a plausible outcome of current research into autonomous agents. Anthropic's CEO, Dario Amodei, said that unless governments act now, we risk ceding control to machines we cannot understand, let alone stop.
Britain has a unique window to lead on this. The government recently hosted the world’s first AI Safety Summit at Bletchley Park, and is now drafting a bill on AI regulation. But the legislation is too narrow. It focuses on immediate harms like bias and privacy, but ignores the existential risk of losing human agency. Anthropic’s paper calls for a new legal category: 'critical AI systems' that must always have human-in-the-loop. These systems would include autonomous decision-making in warfare, critical infrastructure, and healthcare.
The challenge is that the tech industry is moving fast. OpenAI, DeepMind, and others are releasing models that can plan, reason, and execute tasks. GPT-5 is rumoured to have agentic capabilities. Without a binding global framework, we could see a race to the bottom where safety corners are cut for market share. The European Union’s AI Act is a step but has loopholes and lacks enforcement for frontier models. Britain could broker a new treaty, akin to the nuclear non-proliferation treaty, but for AI.
I’ve spent a decade in Silicon Valley. I’ve seen how quickly the utopian rhetoric curdles into dystopian reality. The user experience of society is at stake here. Every new algorithm shapes our digital lives, from the news we see to the jobs we hold. If we hand over the keys to machines without safeguards, we will not just lose privacy. We will lose the ability to shape our own future. Anthropic is right to sound the alarm. But warnings are not enough. We need enforceable laws that require human oversight, transparency, and the right to switch off. Britain has the diplomatic heft and technical expertise to lead this effort. It should act before the machines make the decision for us.
